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Preface 


This publication is the fourth work on the subject of integrated science education in the Unesco 
series The teaching of basic sciences. The first two volumes in the series (New Trends in 
Integrated Science Teaching, Vol. I and II) attempted to exemplify and to analyse world-wide trends 
in this rapidly-evolving field. The third volume addressed itself to the crucial problem of the 
education of teachers for integrated science. It was based on the proceedings of a conference on 
this subject organized by the International Council of Science Unions (ICSU) Committee on the 
Teaching of Science, with Unesco support, at the University of Maryland (United States) in April 1973. 

This present volume attempts to tackle another area vital to the development of integrated 
Science education, namely that of evaluation. This area was singled out by the Maryland Conference 
as one where little information was available and which required urgent attention. Unesco has also 
received many requests for reports of on-going evaluation projects in this field and for information 
on the methodology of testing and examining in integrated science. It has, in addition, been asked 
to assist in designing evaluation programmes, usually in situations where there are severe limitations 
on funds and personnel. 

Accordingly, working in collaboration with the International Council of Associations for Science 
Education (ICASE), Unesco planned a two-part programme activity, consisting of a Symposium on 
"Evaluation of Integrated Science Education" and a publication based on the proceedings of this 
Symposium. A consultant, Dr. D. Cohen of Macquarie University (Australia), was invited to 
assist in the preparation of the Symposium, including the selection of participating specialists, and 
to edit the final publication. 

The Symposium was held at Oxford (United Kingdom) for a four-day period in December 1975, 
concurrently with the General Assembly of ICASE. A series of fourteen manuscripts had been 
commissioned for the Symposium and their authors first met together to discuss and comment on 
each others! contributions. They then met with the ICASE delegates to discuss the relevance and 
applicability of their work to the problems and conditions of integrated science teaching in the 
thirty-eight countries represented. Following this, the authors met together again to attempt to 
incorporate the feedback received and to bring the manuscripts together to form a unified whole. 
While complete unanimity was found to be impossible, a very considerable measure of agreement 
was achieved. As a result of the feedback, an additional chapter on "Evaluating Students! Progress 
in Integrated Science - Public Examinations* was commissioned. 

The publication seeks to present a thorough review and exemplification of current knowledge 
and practice in the field, and the authors have attempted a wide coverage of material and ideas. 

The first part of the book consists of chapters which review and analyse distinct aspects of the topic, 
as revealed by the literature throughout the world and the accounts of experimental projects and 
international, regional and national conferences. The second part comprises seven case studies 

of on-going evaluation projects in different parts of the world. These include accounts of the 
problems encountered when attempting to evaluate integrated science education in a concrete situation 
and some suggestions for tackling these problems. 


The readership for which this volume is intended is a wide one. It is hoped that it will be 
found useful by practising teachers and teachers-in-training as well as by teacher educators, science 
curriculum development and research workers, and Ministry of Education officials. It is suggested 
that selected portions of the book be used as resource material in science education workshops and 
conferences organized by professional bodies concerned with science education. 

Appreciation is expressed to the International Council of Association for Science Education and 
especially to its secretary, Mr. D. Chisman, who undertook the organizational work for the Oxford 
Symposium. Grateful thanks are due to all the authors of commissioned manuscripts and, in 
particular, to Professor J.F. Kerr who responded to the request to write a manuscript at very 
short notice. Particular acknowledgement is made of the devoted work of the Editor. Dr. D. Cohen. 
Tribute is also due to Mr. R. W. Morris of the University of Sussex and his colleagues Mrs. Brenda Harmer 


and Mrs. Brenda Haw who worked together as a team to convert the manuscript into camera-ready copy 
for the printer. 


The views expressed are the responsibility of the authors, and do 


not necessarily reflect the views 
of Unesco. 
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1 Evaluation in integrated science teaching 
—an introduction 


David Cohen 
Macquarie University, Australia 


SUMMARY 


The growing interest in integrated science teaching has sharpened the need to clarify the meanings 
and interpretations of integration and accelerated the development of new strategies of evaluation 
in integrated science teaching. This introductory chapter briefly reviews some alternatives and 
indicates the nature of the chapters to follow. 


INTRODUCTION 


The purpose of this publication is to provide a state-of-the-art report concerning current thinking 
and new trends in evaluation related to integrated science teaching. With the renewed rethinking 
about the nature of integrated science teaching and a renewed interest in reviewing alternatives 
concerning the nature and roles of evaluation, the marriage of evaluation with integrated science 
teaching is very much in the honeymoon stage. Consequently, in many respects, discussions 
remain necessarily pioneering if not speculative. The current evolutionary stage represents one 
good reason for producing a status document just three-quarters of the way through the twentieth 
century. Are there any laws, panaceas, recipes or generalizations emerging about evaluation in 
integrated science teaching? 

Unlike the exact sciences, education can boast no National Standards Laboratories. 
Universal agreements and enactments about education and its terminology do not exist. Even 
the number of definitions for a particular educational expression continue to proliferate and each 
definition may be subjected to widely differing interpretations. The range of alternative 
interpretations thus generated expands exponentially. A first series of decision-making processes 
is concerned with a need to develop or choose definitions. 

In considering Evaluation in Integrated Science Teaching, a detailed review of the terms 
"evaluation" and "integrated science teaching® could fill the whole volume. How might we define 
each? How may we interpret each? What varying influences do varying international contexts 
have upon them? What is the effect of interrelating the terms? What alternative strategies 
for evaluation are available? How might they be implemented? The resultant semantic 
philosophic and educational analyses produce numerous options. The clarification of the conce ts 
of integrated science teaching and of evaluation is akin to the elusive mirage, for as we don 
forward, the obvious answers recede, only to reveal new and broader issues and options. 
Beneath the surface of an initial apparent clarity of the terms lie complex networks which make 
attempted generalizations about "evaluation in integrated science teaching" highly speculative 


1 
Acknowledgements are made to Ms Ruth Cohen, Dr. Barry Fraser, and Dr. R 
, è , . Robert Preci 
for their helpful criticisms. ux 
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Integration 


Consider first the spectrum of acceptable alternative meanings of integration? as the term relates 
to science teaching. ^ "Integration" is generally applied to the relationships within and between 
subjects more often taught separately, so that it refers to a horizontal relationship within a 
curriculum at a particular educational year level. A key intent of those supporting the concept 
of integration is to provide a unified view. An alternative view of integration is to consider it 
as a recombination of subjects or other units into which the whole curriculum was earlier divided 
in order to reflect, in part, the existence of subject disciplines as areas of study, research and 
activity. 

Following an extensive review of the literature, d'Arbon summarized in this way: 


a ‘Integration’, when applied to science courses, means that the course is devised and 
presented in such a way that students gain the concept of the fundamental unity of science; 
the commonality of approach to problems of a scientific nature; and are helped to gain 
an understanding of the role and function of science in their everyday life, and the world 
in which they live. 

An integrated science course eliminates the repetition of subject matter from the various 
Sciences and does not recognize the traditional subject boundaries when presenting topics 
or themes. 
Integrating principles are intended to produce a course which 
(i) is relevant to student needs and experiences; 
(ii) stresses the fundamental unity of science; 
(iii) lays adequate foundations for subsequent specialist study; and 
(iv) adds a cultural dimension to science education.* 


This interpretation reflects the view that the several sciences are characterized by a common 
methodology and that their contents represent a whole and become more meaningful when 
interrelated. This implies that the subdivision of the whole curriculum into separate subjects 
may be considered as "disintegration, as may also the subdivision of science into the separate 
Sciences. Integration may then be regarded as the removal of boundaries between subject 
subdivisions or categories. The extent of integration might vary from the re-combination of two 
subjects into a single subject (e.g. aspects of physics and chemistry into physical chemistry) to 
the re-combination of several subjects into a single subject (e.g. biology, chemistry, earth 
Sciences and physics integrated to form "integrated" science). More widely again, by integrating 
also some concepts or materials from the non-sciences, "integrated studies" would result. 

However, there is also a strong school of thought favouring the retention of separate science 
subjects to reflect the "disciplines", based partly upon the view that people need conceptual 
Structures to avoid a confusion of facts. Such structures are said to be based upon the integrity 
both of content and of the specific approaches to inquiry unique to each discipline. Proponents 
of the disciplines perceive integration as an undesirable trend, if not a vain hope. They argue 
that the separate study of chemistry, physics, biology and earth sciences as scientific disciplines 
1$ necessary to understand the content and methodology of each, and that attempts at integration 
represent an erosion of intellectual integrity. It could be further argued that the separate 
Sciences would lose their own individual identities and so become unrecognizable within 
"integrated science" and that complete fusion (e.g. to create themes) would make it difficult to 
identify and study adequately the separate disciplines within integrated science teaching. 

In summary, consensus is lacking concerning the ®pros® and "cons* of the study of science 
as comprising "separate subjects", or as "integrated science". Criteria to assist decision-makers 
Should include the results of careful appraisals of the context, over-all curriculum objectives, 
functions and availability both of appropriately educated teachers and of physical resources. 


i 
J.A. d'Arbon, "A Study of the Concept of Integration of Science Subjects in Secondary Schools". 
Unpublished thesis presented as 


z partial requirement for the degree of Master of Science, 
Macquarie University, North Ryde, N.S.W., 1972. 
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Evaluation 


The range of platforms related to "integration" is parallelled in number and diversity by the range 
of views related to evaluation. The growing awareness of the complexities of evaluation were 
highlighted by discussions at the Paris Symposium during the preparation of New Trends in 


Integrated Science Teaching : Volume II^ (Richmond, 1973) and again during the conference at 
Maryland on "The Education of Teachers for Integrated Science - Teaching Science for Today's 
Science" related to New Trends in Integrated Science Teaching : Education of Teachers? 
(Richmond, 1974). ^ These led to convene the 1975 Oxford Symposium, devoted entirely to a study 
of evaluation applied to integrated science teaching. 

The study of evaluation in general has been the focus of a rapid growth of international 
interest, mirrored worldwide in the meteoric rise in the number of papers published and the 
number of conferences held concerning evaluation. An index to this growth is the proliferation 
of publications concerning various aspects of evaluation, especially in the last decade. As 
examples, there have appeared numbers of anthologies, a series of AERA Monographs, a 
sourcebook, an encyclopedia, a yearbook, a handbook, a Commission report, a review of research, 
a treatise, a host of books, and scores of journal articles, including whole theme issues 
(see bibliography for details). The reasons for this growth of interest and activity are diverse. 
They include increased funding for education with the related pressures for accountability for the 
expenditure of these funds, a growing disenchantment with the lack of pay-off from traditional 
methods of evaluation, the generation of a new range of strategies for attacking the problems of 
evaluation, increased availability of professionals with competencies to handle evaluation studies 
and an emphasis on the improvement of educational quality and verification of this improvement 
through the provision of evidence. 

The expansion of interest in and views about evaluation have led to polarizations of views on 
a number of dimensions describing the nature, scope and strategies of evaluation. Table 1 
represents the range of views along eight different dimensions. 


1 Peter E. Richmond (ed), New Trends in Integrated Science Teaching : Volume Il, Unesco, 


Paris, 1973. 


2 Peter E. Richmond (ed), New Trends in Integrated Science Teachin; Education of Teachers, 


Volume Ill, Unesco, Paris, 1974. 
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Table 1. Some dimensions in evaluation. 
Dimensions Range of views represented along continua 
Data provision to Data are pre-selected by Broad, diverse arrays of data are 
decision-makers evaluators to provide to provided, from which decision-makers 
decision-makers make their own choices. 
Nature of data Normative data;  value-based Descriptive data;  value-free 
(e.g. examination scores) (e.g. classroom anecdotal records). 
Referents for Numerical or letter grades used Student progress related to objectives 
evaluation of to compare student performances (criterion-referenced). 
student progress (norm-referenced) 
Bases of Heavy emphasis upon instruments Greater reliance upon teachers who 
evaluation to provide standardized test exercise spontaneous subjective 
Scores ` judgements. 
Specificity of Aspects to be evaluated fully Emerging developments result in 
aspects evaluated specified (often behaviourally) progressive modification of evaluation 
and pre-determined (means-end strategies (including evaluation of 
evaluation) unintended outcomes). 
Breadth of Evaluation limited to assessment Evaluation embraces assessment of 
evaluation of student achievement students, teachers, curricula, 
learning processes, materials, 
teaching skills, etc. 
Process/product Evaluation concerned with ends Evaluation concerned also with 
evaluation only (i.e. products) processes of development, 
implementation and evaluation. 
Phasing of Formative and summative Evaluation seen as continuous, 
evaluation evaluation considered as overlapping and interacting processes, 
two-phase, separate and including reflective evaluation as well 
comprehensive scheme as formative and summative evaluation. 


e a G l ee 


, Each dimension in Table 1 represents a continuum of views. For example, the first 
dimension relates to the provision of data to the decision-makers. The views along this 
continuum range from those who consider that evaluators make a pre-selection of which data to 
provide, to those evaluators who consider that a mass of diverse data Should be provided from 
which the decision-makers choose. 

As used in everyday speech, evaluation is concerned with judgements about values, with 
questions of the nature: "How good is ...?" and "What is the worth of ...?*. However, 
these has been a growing tendency to separate the judgemental from the information-provision 
aspects of evaluation. Thus, evaluation has been defined in this way: 
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" Educational evaluation is the process of delineating, obtaining and providing useful 
information for judging decision alternatives? E 


This view of evaluation implies that it is the prime tool for decision-makers who include the 
students, teachers, parents and community, education departments, governmental agencies, 
project teams and publishers. Not only is evaluation concerned with gathering, analysing and 
reporting data in order to improve decision-making, but evaluators may also help decision-makers 
to articulate their value positions, without the evaluators themselves becoming decision-makers, 
as Tawney contends in Chapter 2. 

Evaluation may be considered as part of a complex network, interacting with the context 
and functions which the evaluation is to serve, as well as the decision-making processes 
(see Fig. 1). The potential functions to be served by the decisions to be made should help 
evaluators decide upon the most appropriate data to be gathered. Figure 1 shows 
diagrammatically the complex network of interactions between the concern of evaluators to service 
the decision-makers with data appropriate for their differing contexts and functions. 


Fig. 1. Evaluation as a complex interacting network. 
CONTEXT FUNCTIONS 
e.g. 
e.g. . E FACTORS INFLUENCING 
national project, revision, EVALUATION STUDY 
classroom, adoption, 
developing nation. cost analysis. 
^a. 
EVALUATION 
gathering data; a 
analysing data; 
reporting data. DECISION MAKING <= 
reviewing alternatives, 
selecting actions. w 
1 


Daniel L. Stufflebeam, et al. Educational Evaluation and Decision Making, F.E. Peacock 
Publishers Inc., Itasca, Ill., 1971. 
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The context of evaluation will heavily influence the strategies selected. For example, the 
methods of evaluation will differ according to context. Contexts will vary with the type of 
society (with its implicit values), the economic background of schools, the availability of adequate 
funding and personnel, the extent of centralization and the stage of development reached in the 
educational system (e.g. level of teachers). 

Another kind of variation in context relates to the magnitude of the evaluation activity. 

In the case of a large-scale evaluation (as might be needed for a national, state or curriculum 
project activity), members of the development team and evaluators may each have differing 
perceptions about evaluation, and differing commitments to different approaches. Factors such 

as interpersonal relationships and perceived status of the opinion-holders may be one influence 
upon the acceptance or rejection of their views. Judgements may be periodic, formal (e.g. based 
on standardized or external testing) and "objective". ^ An example of small-scale evaluation is the 
teacher in a classroom, making continuous, informal, spontaneous and subjective judgements. 

Just as evaluation procedures are likely-to differ in differing contexts, so too are they 
likely to vary according to the range of decisions to be based upon them. The functions of the 
decisions may be diverse, relating to such questions as whether to revise aspects of a curriculum 
in its developmental stages, whether to change approaches to implementation, or to adopt/adapt/ 
reject some "package" approach to curriculum materials or teaching strategies. Criteria upon 
which such decisions might be based include costs, predicted educational benefits and feasibility. 
These decisions may vary also according to the context in which they are made. 

The complexities of the interactions represented in Figure 1 make it clear that there can 
be no unique approach, no sets of rules or recipes or "pat" answers to problems of evaluation. 
Even when a set of strategies is selected, these themselves are likely to change as the evaluation 
activities progress. What this present volume seeks to do is to elucidate some of the 
alternative approaches to evaluation in integrated science teaching. 


Evaluation in Integrated Science Teaching 


In planning this volume, an attempt was made to produce an exhaustive and logically comprehensive 
review of evaluation related to the various aspects of integrated science teaching. This 
necessitated contributions which analysed the strengths and limitations of evaluation (Tawney) and 
examined how evaluation could be used to improve the quality of decisions (Welch). Evaluation 
may be applied to a variety of aspects of integrated science teaching. In this volume, successive 
chapters describe the evaluation of integrated science curriculum materials (Bloch), of teaching 
Skills (Sutton) and of students through the use of assessment instruments (Mayer and Richmond). 
Subsequent chapters review the use of teacher-developed approaches to evaluation (Ramsey) and 
assessment by means of external examinations (Kerr), and these are followed by Harlen's account 
of how evaluation may be used as the basis for adapting integrated science teaching experiences 
to individual pupils. The six national case studies which follow were selected to provide diverse 
oe of influences of differences in contexts and functions of evaluation in integrated science 
ching. 

As most of the approaches, techniques and examples contained in this volume are relevant 
to other areas as well as to integrated science teaching, it is hoped that the reviews which it 
contains will prove useful to both science teachers and evaluators in widening their educational 
horizons. May the volume itself be evaluated through the improved opportunities it might open, 
at least through integrated science teaching, for individual learners throughout the world. 
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2 The evaluation of integrated science 
education — its strengths and limitations, 
its scope and methodology 


D. A. Tawney 


University of East Anglia, United Kingdom 


SUMMARY 


Evaluation is undertaken to improve education, to reassure and, regrettably, because it is 

currently fashionable. Its strengths come from separating information-providing and option-probing 
from decision-making roles; its limitations come from the over-expectation it so often produces. 
The areas about which information should be provided include context, aims, processes and 

effects. Its methodology needs to be wide-ranging and is likely in practice to lie between the 
descriptive and quantitative extremes described. 


Why Evaluate? 


Through the rhetoric of justification emerge three main answers to the question: Why evaluate? 
Firstly, it is hoped that evaluation will help to improve education, whether it is used 
formatively to help a curriculum project develop its materials or to help a teacher develop his 
own teaching or, summatively, guiding a decision about which curriculum to adopt. After a 
decade and a half of accelerated curriculum development, education, including science education, 
has not changed as much as many had hoped. Evaluation is part of an attempt to go about 
development more sensitively and critically; evaluation is for quality control. 

Secondly, evaluation is used to defend and to reassure. Funding bodies need confirmation 
that money has been well-spent, governments that a curriculum is worth disseminating, head 
teachers that pupils are not becoming "innocent victims of the poor pilot scheme" (as described 
for Malaysia by Sim, chapter 15) and the curriculum developers themselves that they have 
achieved something. Such a use of evaluation is understandable and at times necessary, but it 
should be regarded with caution as there is a danger of devoting resources to a study which will 
have little or no effect on decisions. 

Thirdly, evaluations are commissioned because at the moment evaluation is fashionable; 
no good curriculum project can afford to be without one! It is not of course impossible for a 
study commissioned for so superficial a reason to turn out to be very valuable but unless an 
evaluation is commissioned carefully, with a full understanding of its purpose and likely outcomes, 
results can be disappointingly irrelevant. 

The question; Why evaluate? is misleading, for everyone evaluates anyway. The 
question is whether this natural and inevitable aspect of most purposeful activities is better 
formalized, recognized as a separate activity, made the responsibility of particular people. — 
Does this formalization improve the decisions made? Evaluators have to confess that there is 
as yet little evidence that it does. 

Against evaluation as a separate activity, it must be argued that it is expensive; a 
summative evaluation may cost 5 per cent of a project's budget, a combined formative and 
summative evaluation 10 per cent. To spend less is to risk wasting money; for example, from 


2 Scottish experience Jeffrey points out the limitations of using cheap labour involving students 
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working for higher degrees. Another argument is that too much information hinders 
developers' creativity, particularly if their confidence is sapped by unfavourable reports. 
However, although evaluators must be aware of these dangers, the argument represents a 
short-term view. 

Readers must decide for themselves whether evaluation should be a formalized, separate 
activity. The subsequent pages, particularly the case studies, are intended to help this decision. 


What are its strengths? 


From the definition in Chapter 1, it is clear that evaluation as a separate activity is to do with 
the provision of information, with the intention of helping those who have to make decisions to 
make them more rationally. By separating the information provision from the decision-making, 
it is hoped that the information may be made more reliable. Evaluators are less involved, less 
committed to the materials or processes being studied, less anxious about the consequences of 
decisions. They have a more detached perspective and different values, more time for gathering 
information; teachers will tell evaluators things which are different from those they will tell the 
developers and the reports of the evaluators will have greater credibility. 

Of course, it is quite possible for the developers themselves to adopt evaluation roles, 
deliberately distancing themselves from their creative roles; instruments such as tests and 
observation schedules are a good way of achieving this. It was the policy of the Nuffield 
Advanced Biological Science Project to share evaluation roles out among the developers 
but small projects will do this of necessity as must teachers trying to evaluate their own work. 
However, it is difficult to play two roles simultaneously and the individual teacher may well be 
advised to seek the help of a colleague (see Chapter 5). 


The value of a separate evaluator becomes clear if, following Stufflebeam 2, it is accepted 
that an evaluator should be concerned not only with the provision of information about what is but 
also with helping decision-makers articulate their values and explore future options, what might be. 
Many would agree with Sim that brainstorming with the decision-makers is part of evaluation. 

The role of evaluator has opposite tendencies to that of the decision-maker; it encourages 
divergence and so deters a decision-maker from converging too soon, and it seeks to publish 
information, not trusting it unless it has been subject to public scrutiny, while decision-makers 
often prefer secrecy. 

The ultimate strength of evaluation depends on a belief that decisions are better for being 
based on the maximum of relevant information and for being made after alternatives have been 
properly explored. It is the evaluator's job not only to supply this information but also to supply 
it at the right time and in a readily comprehensible form. 


What are its limitations? 

If evaluation is regarded as a panacea for all educational ills, it is bound to produce disappointment. 
Evaluation too often raises expectations which are too high; the following section is an attempt to 
lower them. 

Evaluation in education cannot provide information with the reliability of the natural sciences. 

This is contrary to current popular belief, and to that of evaluators in the past, who confidently 
adopted natural science research methodology. Evaluators are divided on whether the failure of 
past studies based on this methodology should lead to more sophisticated adaptations of it or to 


its abandonment. Even if numerical techniques were perfected, evaluation would still not be as 
decisive as some might hope because the aims of education depend on value judgements. 


lp J. Kelly, "Nuffield A-level Biological Science Project", in Evaluation in Curriculum 
Development: Twelve Case Studies (Schools Council Research Studies), Macmillan, London, 1973. 


2 stufflebeam , et al, Educational Evaluation, op. cit. 
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The information provided by educational evaluation is unlikely to over-ride direct experience. 


Jolm Glashan 


Teachers, administrators and developers build up direct experience by themselves observing and 
talking to participants. Reay's comment (Chapter 11) that there ... "is little evidence that 
the data were used by the decision-makers" ... is fairly universal; direct experience tends 

to win against a formal report. Evaluators need to interact with the decision-makers, 
"brainstorming" and producing more readable reports with provocative covers (as Sim suggests 
- Chapter 15), if they are to compete on equal terms with direct experience. 


Evaluation cannot ensure that decisions are taken on educational grounds alone. 


s and politics play a big part. "No amount of evaluation ... will take the place of a 
dissemination base" (Reay) in securing adoption and some administrators will push ahead with a 
project, whatever the evaluation findings, as resources have been committed to it and their 
prestige is at stake. A wise evaluator tries to avoid providing information for a decision that 
has already been made, perhaps shifting to a related one which is still open. 


Evaluators ought not to make decisions for the decision-maker. 


Some decision-makers hope that they will but most evaluators believe that it ought not to be part 
of their job; the hope comes from a belief that statistics will lead automatically to clear-cut 
decisions. This is not the case, as decisions are seldom clear-cut. Decisions involve 
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balancing values. Evaluators have to make decisions at a different level in choosing techniques 
and date. 


Evaluation will not necessarily make decision-making easier. 


Evaluation challenges the fixed view by opening up ranges of things to be considered, of possible 
alternatives. This can make decision-making harder although hopefully the decisions will be 
better. Nevertheless, some decision-makers will welcome the clarification, the marshalling of 
pros and cons, which a well-structured report offers. 


There is no agreed evaluation methodology. 


The trend papers in this volume reveal a range of views on evaluation theory and the case studies 
a diversity of practices. It is difficult for the customer to be quite sure what he is buying! 


A misconception of evaluation. 
A common misconception of evaluation is to regard it as: 


‘essentially the process of determining to what extent the educational objectives are actually 
being realised by the program of curriculum and instruction. However, since educational 
objectives are essentially changes in human beings ... then evaluation is the process for 
determining the degree to which these changes in behaviour are actually taking place.' 


This definition is embodied in the Eight Year Study directed by Tyler in the 1930s? Its dependence 
on behaviourist psychology and the psychometric tradition is apparent. 

The influence of this definition has persisted over the intervening decades, although evaluations 
which over-relied on it have frequently failed in a number of ways. These are detailed in several 
places 3 and will merely be outlined here. Frequently little or no significant difference is found 
between the changes, cognitive and, particularly, affective, of experimental and control groups * 
(Reay); such differences which do occur do not provide useful formative information; it is 
impossible to control variables (cf, Reay), particularly class-room processes, to the extent the 
approach demands; it is difficult to accommodate informal data although these may influence the 
decision-maker strongly; the need to measure for achievement of objectives tends to emphasize 
those elements which are easiest to measure. 

This style of evaluation, unable to provide data until the end of the experiment, is not 
flexible enough to respond to the changing information needs of a project team; it is not what 
Stake calls responsive evaluation. 

As well as the failure of the achievement-of-objectives evaluation to provide satisfactory 
information, the curriculum model of which it is a part is increasingly regarded as inadequate 9 


law. Tyler, Basic Principles of Curriculum and Instruction, Chicago, University of Chicago 
Press, 1949, p. 105-6. 

2 
F.R. Smith and R.W. Tyler, Appraising and Recording Student Progress, Vol. 3 of Adventure 
jn American Education, New York, Harper, 1942. 


3 stufflebeam et al, Educational Evaluation, op. cit. 


Parlett and Hamilton in D.A. Tawney (ed), Curriculum Evaluation Today: Trends and 
Implications (Schools Council Research Studies), London, MacMillan, 1976. 


tw. Harlen, Science: 5/13 - A Formative Evaluation, London, MacMillan, 1975. 


bm Stenhouse , An Introduction to Curriculum Research and Development, London, Heinemann 
1975, p. 70. ` 
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Innovations are seen by many as likely to be more successful if objectives are not over-detailed 
and pre-defined but allowed to develop during the project. 


The scope of evaluation. 


Most agree that evaluation needs to be wider than the measurement of achievement. Despite 
disagreement on which methodologies to adopt in its place, whether to develop more sophisticated 
statistical designs or to abandon the approach entirely, there is agreement on the scope of 
evaluation and on the areas needing investigation. 


Context. 


Many recent evaluations have included investigations into the context in which the curriculum 
project is working, into existing curricular and class-room practices, into alternative schemes 
which might be imported and adapted and into opinions on possible proposals. Such investigations 
are given a variety of names: reflective evaluation 1, presage evaluation (Sim), antecedent 
evaluation? (cf, Yoloye, Chapter 10). 

Evaluators have looked at material resources such as laboratory apparatus; Sim describes 
such a study. Some have looked at the qualifications of the teachers involved. Other important 
contextual variables which may need investigation are the school timetable and the public 
examination system;  Krasilchik refers to the difficulty of introducing a course which does not 
lead to the all-important university entrance examination. Sim also refers to the effects of a 
tradition of public examinations. 

Evaluators seem to have been less ready to examine less tangible but perhaps fundamentally 
more important factors. Integrated science projects often advocate 'discovery methods! in which 
teachers act more as guides than authoritative sources of information; stepping down from the 
platform and across the boundaries of the disciplines may produce insecurity. At a more prosaic 
level, it may reduce prospects of promotion (Krasilchik, Chapter 14). Yoloye, looking at the 
Stage of development of African primary schools, uses Beeby's classification to cast doubts on 
the compatibility of the African Primary Science Project with current teaching methods. 

The problem is explored more thoroughly in an area with similar problems, namely the 
integration of the humanities, by Shipman? in a Study which is of relevance to all curriculum 
developers, particularly those involved in cross-disciplinary projects. 

: How contextual information is used depends on the methodology adopted: in some the context 
is one source of the variables selected for measurement at the beginning of the study and also of 
some of the clues for interpretation of findings at the end; in others, the study of context extends 
throughout the study, its interaction with the innovation providing increasing insight. 


Aims. 


Traditionally, evaluators have been expected, as a pre-requisite to designing tests and schedules, 

to help their projects to clarify and operationalize objectives. More recently, with the increasing 
recognition that objectives emerge, develop and change (cf, Yoloye) throughout a project, they are 
likely to replace pre-definitions with a continual surveillance and attempt to clarify. 


1 


D. Cohen, "Evaluation of Integrated Science Curricula", in Richmond (ed), New Trends, Vol. 2 
op. cit., p. 143-65. 


2 


i Stake, "The countenance of educational evaluation", Teachers! College Record 68, Vol. 7, 
> p. 538. 


3 " š 
M.D. Shipman, Inside a Curriculum Project, London, Methuen, 1974. 
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The aims of the curriculum project must be related to the wider context of the society of 
which the schools in which the innovation is to occur are a part. In countries with a 
well-defined society-orientated educational philosophy, a child-orientated curriculum may appear 
out of place (cf, Yoloye). In many, a wide range of views coexist; in consequence the aims of 
a project will represent a compromise and its organizers must expect its products to be used in 
a variety of ways. 

Sim refers to parental anxieties which reflect a view that science is part of the serious 
business of acquiring qualifications; this suggests the need for those drawing up the aims of a 
project to be aware of vocational pressures; future employers and national manpower planners 
may well be influential. 

At a more detailed level, Kelly made interesting use of teacher ratings of the objectives of 
sixth form biology teaching, comparing their ratings of objectives for desirability and 
attainability. Discrepancies between these, and between these and actual attainment examined at 
different times in the life of the project, provided valuable formative data. Of particular interest 
as an example of responsive evaluation is how a discrepancy between the desirability and 
attainability of the objective 'handling quantitative information’ led to an investigation which 
resulted in a considerable improvement in its attainment. 

An aspect of integrated science which the evaluation may need to examine is the view of the 
nature of science reflected by a curriculum project, particularly through its aims. Two features 
make a careful examination of the view of science important: one is because an integrated science 
project frequently emphasizes process rather than content, the other because of its justification for 
integration, whatever is meant by this. 

The problem arises in projects for older pupils when an attempt is made to teach a process 
by which science develops, the 'scientific method'. One model of this is "pattern-finding"; 
for example, in the Schools Council Integrated Science Project (SCISP), "pupils are asked to 
extract a pattern from the information"? gained from experiments they have performed. This 
process, called induction, has been discredited since the eighteenth century; for a review of the 
arguments against it, see Tawney, Jevons, or Medawar. 3 Although there is no consensus on how 
Science progresses (except that it is not by induction!), it is a pity that more projects have not 
placed at least equal emphasis on a hypothetico-deductive approach in which pupils use their 
existing knowledge to suggest hypotheses which they then test experimentally. With this approach, 
it is easier to show, first, the essential part which existing knowledge plays in discovering new 
knowledge, secondly, that science is a human creation and, lastly, the strengths and limitations of 
Science. 

In a similar and closely related category is the rationale behind the notion of 'integration' 
used in a project. This has been explored elsewhere |, but in suggesting this as an aspect of 
every project which may need evaluation, it is necessary to warn evaluators that some of the 
higher sounding reasons given for integration frequently do not appear to be embodied in the 
learning experiences advocated for reasons that are not always clear. Perhaps interviews with 


l kelly, op. cit., p. 95. 


2y. Hall, Patterns: Teachers' Handbook, London, Longmans and Penquin, 1973, p. 30. 
3D.A. Tawney, "The nature of science and scientific enquiry" in Sutton, C.R., Haysom, J.T. 
(eds), The Art of the Science Teacher, London, McGraw-Hill, 1974, p. 19-29; F.R. Jevons, 
Science Observed, London, George, Allen and Unwin, 1973; P.B. Medawar 


; "Hypotheses and 
Imagination", in The Art of the Soluble, London, Penguin, 1969, p. 147-73. 


ud Rutherford and M. Gardner, Integrated Science Teaching in New Trends, Vol. 1, 1971, 


Op. cit., p. 47-55. Zu 
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the project team may clarify this area and prevent resources being devoted to an empirical study 
of the achievement of aims whose statement was never meant to be more than educational 
rhetoric. Brown's analysis later in this volume of some affective objectives in the Scottish 
Integrated Science Project and the arguments given for including these is an example of one 
approach to this area. 


Learning Processes. 


What learning processes or transactions does a project advocate and how do they differ from 
traditional practices? And what practices do teachers ostensibly following a project, adopt? 
These questions are framed in terms of a curriculum project; there are corresponding ones 
for teachers evaluating their own work. 


On what psychological theories are the experiences advocated by a project based? If it is 
adopting a 'discovery' approach, has it considered the objections to it (e.g. Ausubel 1)? Is the 
project team aware of the limitations of learning theories (e.g. those of Piaget and Gagne), 
indeed of social research in general? ? Although the bulk of evaluation in this area will occur 
at the start of a project, evaluators must realize that attempting to implement learning processes 
will often give insights into the theories behind them. 

To move from central policies out into the field, what are the teachers' understandings of a 
project's philosophy? For example, Sim describes a study which suggested that teacher 
conceptions of the term 'discovery' were limited. 

What discrepancies exist between what teachers say they do and what they do? Increasingly, 
emphasis in evaluation is shifting to classroom observation, for there is agreement that the failure 
of curriculum projects to change science teaching by the extent hoped is due not so much to the 
failure of the classroom processes advocated as to their never really being practised. Jeffrey's 
report that "... many teachers were not attempting to teach towards these attitudinal objectives" 
is more generally true (cf, Yoloye). 

This suggests that the failure lies not in curriculum strategy but in innovation strategy 
- the way a project interacts with teachers and indeed with the whole education system - to 
further the adoption of its curriculum strategy. The answer seems to be more and better 
teacher training. 

It is an evaluation task to try by a study of process to separate the degrees of success of 
these two Strategies; the failure of the measurement-of-achievements model to make the 
T p es why it was discredited. Evaluators will respond to the problem in 

ght obtain data using highly structured observation schedules and use 
sophisticated statistical techniques to relate the data to measured effects and other variables. 
Others might try to typify process by a case-study of a lesson and relate it qualitatively to 


effects. In no area of evaluation are the problems more difficult and the attempted solutions 
more diverse. 


Effects. 


The effects of an innovation cover a wider a 
their ee is even more difficult than that of achievement of objectives. 
ntegrated Science courses often precede other science courses and it is in these that some 


oe pet the course may be detected. Science teachers in the United Kingdom who are 
idering adopting the 14-16 SCISP course are likely to be concerned with how pupils who have 


rea than its objectives and are frequently unexpected; 


1 
D.P. Ausubel, "Some psychological considerations i 


n the objectives and design of an 
elementary-school Science program" 


; Science Education, Vol. 47, No. 3, 1963. 
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M.D. Shipman, The Limitations of Social Research, London, Longman, 1972. 
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followed this course will fare in the subsequent 16-18 courses leading to the important General 
Certificate of Education (G.C.E.) Advanced (A) Level examination, so that it is important to look 
for the effects of the new curriculum in this area. 

There is equal need to monitor concurrent effects. An innovation which advocates informal 
relationships between teachers and students, which cast doubts on the value of formal note-taking 
and other traditions and encourages the questioning of authority, may bewilder pupils in a school 
with a traditional style of teaching and may cause sharp resentment from other teachers. However, 
the achievement of cognitive objectives is still an important aim of most projects and most 
evaluators will want to test for such achievement; the means they employ will depend very much 
on the assessment traditions of the country and, if there is a public examination system, the 
relationship of the course being evaluated to it. 

Later in this volume, Reay refers to the value of a properly-designed test as a teaching 
tool, while Sim describes what happens when there is a poor match between the objectives of a 
course and a public examination. 

For courses remote from a public examination and implemented in areas where testing is 
perhaps viewed with suspicion, other means of evaluation of achievement may be more appropriate: 
for example, the Diagnostic Statements developed by Harlen! for the "Science: 5/13 Project" 
represent a checklist of behaviours to enable teachers to monitor pupil progress. Evaluators 
differ in their views on the use of tests in this kind of context; some feel that reliability can be 
achieved only if properly standardized tests are used; others feel that tests specially designed 
to match project objectives will be more satisfactory, particularly for formative purposes, and 
will happily sacrifice reliability for unobtrusiveness and match between course and test. 

Evaluation of affective objectives presents more of a problem. It is difficult to tell whether 
the failure of many existing tests to detect significant gains is due to the tests or to the failure 
of teaching to bring affective changes about. Certainly there seems to be a general tendency 
for pupil familiarity with a school subject to breed, if not contempt, a lessening of enthusiams 
for it^ (cf, Jeffrey, Chapter 12). Attitude scales should be used with caution; some evaluators 
will prefer to replace them or complement their use with other measures such as interviews and 
classroom observations. 


Evaluation Methodology. 


To evaluate the areas indicated above demands a wide-ranging methodology and access to the 
viewpoints of a number of disciplines. Philosophy of science, psychology and sociology can all 
provide perspectives from which to try to order the complexity of an innovation in integrated 
Science. It is clear that no single evaluator can possess an adequate knowledge of all these 
disciplines and therefore it will be necessary to call upon the services of sympathetic experts in 
order to present the project with relevant information. In doing this, it must be remembered 
that presenting too many data will hinder rather than aid decision-making; data must be 
structured into information. 

The evaluator will need to make use of the whole range of evaluation techniques: tests, 
questionnaires, classroom observation, teacher diaries, interviews, meetings (cf, Krasilchik, 
Jeffrey). The degree of structure used in questionnaires and interview schedules will depend on 
conditions; for example, Yoloye reports that the African Primary Science Programme found 
Flander's observation schedules too complicated; thus simpler ones were developed. Resources 
have a strong influence and clearly a teacher evaluating a small curriculum development in a 
school is likely to use fewer measures than is a nationally-funded project. However, even 
well-funded projects have to count the cost; many evaluators would agree that pupil interviews 


1 Harlen, op. cit. 
2 
ibid. 
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are a very informative and flexible technique, but it has been estimated that interviews may cost 
twenty times as much per pupil as a pencil-and-paper test. Teacher questionnaires are a cheap 
way of obtaining data but it is difficult to get a reasonable percentage returned and those which 
are returned have not always been critically completed, as Reay points out. 

In addition to these techniques, the evaluator will want to use what Eraut! (in Tawney, 1976) 
calls intrinsic evaluation, that is, an examination of the project documents, statements of policy, 
records of meetings and materials to conduct an exploration of the assumptions, values and 
agreements which lie behind a project's proposals;  Yoloye gives an example. 

Evaluators disagree on the extent to which data provided by these means should be quantified. 
Particularly for a summative evaluation, some evaluators prefer a highly statistical approach 
(for example, multivariate analysis), in which all data are quantified and analysed to obtain the 
significant variables and to identify the relationship between them”. However, this is a 
cumbersome method and cannot provide information rapidly on the problems as they arise. 
significant variables and relationships to emerge, the appropriate variables have had to be 
recognized from the start. Even if relationships are detected, their interpretations often remain 
highly speculative. 

This author favours the more descriptive illuminative approach of Parlett and Hamilton 
which has its roots in social anthropology. This characteristically has three stages: 
‘investigators observe, inquire further and then seek to explain'. General observation of a 
project in action both at its base and in the field will suggest recurring trends and issues. Some 
of these will be selected as the subject of more sustained and intensive inquiry with observation 
more directed and systematic. The third stage consists in 


For 


- Seeking general principles underlying the organization of the program; spotting 
patterns of cause and effect within its operation; and planning individual findings within 
a broader explanatory context",? 


The flexibility of this approach enables the evaluation to focus on problems as they arise. Some 
evaluators may accept this but see on the debit side a considerable loss of rigour. As discussed 
earlier, whether or not traditional evaluation designs really have greater rigour is debatable, but 
certainly methods must be employed to make evaluation more rigorous than casual observation. 

The methods used in the illuminative approach to obtain rigour are triangulation, 
re-iteration and exposure. By triangulation is meant the use of several methods to expose an 
issue or support an hypothesis. Student opinion revealed through interview may by itself be 
unreliable, but supported by the observations of teachers and the results of informal tests, may 
provide sufficiently reliable information. To employ other methods could be regarded as 
over-kill. 

Re-iteration is the process of exposing early descriptions and interpretations of what is 
Occurring to criticism by others and subsequently redrafting them for further exposure. Project 
newsletters or regular evaluation reports are one means of exposure and they also help to 
involve teachers in evaluation and to encourage an evaluation 'atmosphere' (cf, Reay). 


Evaluation findings. 
The va pm approaches of these two methods of evaluation are manifest in their reports. The 
report of a summative evaluation with a statistical emphasis is likely to say little about the 


Ee Er E used for formative purposes; it will contain sections concerned with the 
esign of the evaluation, the analysis of the data, the findings (weightings, correlations, degrees 


1 
Tawney (ed), Curriculum Evaluation Today ..., op. cit 


W.W. Cooley and P.R. Lohnes, Multivariate Data Analysis, New York, Wiley, 1971. 
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Parlett and Hamilton, op. cit., p. 93. 
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of significance) and the interpretations of the findings. Whether or not statistically significant 
relations are established depends on the inclusion of the right variables at the start. Whether 
or not educationally significant information is obtained depends on other factors; too often the 
findings are either obvious or concerned with variables outside the control of the decision-maker. 
As was mentioned earlier, interpretations have often to be very tentative as they are made too 
late to be tested. 

The report of an illuminative evaluation is less likely to distinguish between formative and 
summative evaluation. It will probably be chronological, as the collection of evidence and the 
interpretation of evidence occur simultaneously and these are closely linked with the resulting 
decisions. 

The final report will rely heavily on reports issued during the life of the project in order to 
give the decision-maker a "... description of the natural history of our conclusions ... an 
opportunity to make his own judgements" |. Because there is seldom consensus on judgements, 
it is likely that the report will give several judgements from different participants in the project 
and from outsiders who have had contact with it. 

This approach to evaluation is flexible but it is in practice difficult to obtain sufficient 
triangulation to achieve rigour; furthermore, although evaluators may publish their tentative 
hypotheses in a newsletter so that they are subject to critical scrutiny, they may get little 
response. The ability of this method to provide all the information needed depends on the 
sensitivity of the evaluators to issues as they arise and their ability to organize their resources 
So that they can respond to them. 

Perhaps more important than the limitations of these two approaches is a lack of knowledge 
not so much of what data to collect and how to collect them but of how to present the 
decision-makers with balanced information in a form which will command their attention and 
influence their decisions as much as their selective but direct experiences of the Project. 

Some evaluators are experimenting with more and more arresting forms of presentation; the 
Malaysian Case Study presented by Sim gives an example. 


Conclusions. 


In practice, many evaluators occupy a position somewhere between the two extremes described, 
as exemplified by the case studies in this book. The case studies use a range of methods 
according to their resources and opportunities, aiming for Sim's 'continuous comprehensive, 
concerted and co-ordinated' 'Orchestra Model' but probably achieving his 'Jigsaw Model'. 
Hopefully there will not be too many missing pieces! 
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SUMMARY 


Curriculum developers, teachers, and funding groups face a number of decisions concerning 
integrated science. Providing information to assist in these decisions is the function of 
evaluation. In this paper, several key decisions are identified and the evaluation literature 
pertaining to these decisions is reviewed. 


INTRODUCTION 


In recent years, there has been considerable interest in removing the traditional subject boundaries 
that have characterized science education. This movement has taken on a variety of names 
(integrated science, unified science, interdisciplinary, etc.) but the common thread has been a 
rejection of the traditional labels of chemistry, biology, and physics in developing new courses. 

In this chapter, the term "integrated science" will be used generally to represent this trend, 
although a more specific definition will be presented later. 

_ The apparent growth of the movement is noteworthy. In 1968, only twenty-five integrated 
Science projects were known to be in operation in junior and senior high schools in the United 
ae pis ced By 1973, the numbers had risen to seventy-three known projects in the 
Safe ea ae fifteen in other countries. By 1975, Showalter and his colleagues at Ohio 

ap à ty were reporting 170 programmes throughout the world which could be called 
unified science programmes. 
the ME ae que on bass degree of integration of science courses can be obtained from 
Develoiantes 7, This re ae ional Clearinghouse on Science and Mathematics Kurs 
A toed ar 24 puofein me c: LS 392 projects developed between 1956 and ns "T 
degree of integration varles E aan on RN NS ai in the 
movement is clearly evident. ý "E praecise SRE Wi 

Thus, this represents an examinable development in international science education. 
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information obtained from careful and effective course evaluations. This requirement for 
information gives rise to the general problem of this chapter. What evaluative information is 
currently available to assist those facing a variety of decisions about integrated science education? 
The specific objectives of this chapter are as follows: (1) to define several key terms - evaluation, 
decision-making and integrated science; (2) to identify several important decisions regarding 
integrated science; (3) to review present evaluative evidence that pertains to these decisions; 

and (4) to offer several suggestions for future evaluation studies in integrated science. 


DEFINITIONS 


Evaluation 


Evaluation is generally defined as the process of delineating, obtaining, and providing useful 
information for judging decision alternatives.! The use of the word judging suggests the important 


place of values in the evaluative process. The decision-maker is required to ascertain the 
relative value of competing alternatives. Evaluation is designed to assist in this valuing process 
by providing relevant, credible and valid information. Thus, evaluation is an information- 


generating process. 


Decision-making 


Decision-making can be considered in terms of both technology and human variables. The former 
stresses quantitative techniques such as those found in management science (e.g. programming, 
scheduling, simulation). These techniques can be used to reduce the degree of uncertainty in a 
decision situation. The latter concentrates on the problem-solving limitations of people. The 
decision-maker is viewed in terms of culture, personality, aspiration and problem solving strategy. 

In this chapter, we will accept a simple view of decision-making where a decision-maker 
faces a known set of alternatives and selects a course of action by a rational selection process. 
However, we must remember that a decision-maker cannot recognize all feasible alternatives and, 
furthermore, is influenced by a biased perception of the environment. Thus, the actual 
cognitive process of deciding is indeed complex. 

The key element in decision-making, as defined here, is choice among various courses of 
action. The major assumption we make is that information will improve the probability of a 
"successful" choice. Success is used in the sense of achieving some desired goal or objective. 
Whenever an individual is uncertain as to how best to pursue the objectives, indecision results. 
Reducing this indecision is the purpose of evaluation. 


Fig. 2. Relationship between evaluation and decision-making. 


Values lic T ad Environment 
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The need for evaluative information arises from the uncertainty felt by the decision-maker. 
The choice of alternative courses of action is a function of the interaction of the decision-maker's 
environment, his value system, the nature of the uncertainty and available information. 


1 stufflebeam et al, op. cit. 
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For example, suppose a teacher is considering changing the physics textbook. A number of 
alternatives are available: PSSC, Project Physics, Nuffield Physics, or a revised traditional text. 
The need for evaluative information regarding these courses is directly proportional to the 
uncertainty felt by the teacher. If the teacher is told what book to use, there is little uncertainty 
and no need for evaluation. If there is freedom of choice, greater evaluation information should 
increase the likelihood of a satisfactory choice. The values of the teacher (including such things 
as perception of the purpose of pre-college physics) as well as such personality characteristics 
as flexibility, need for change or conservatism and real and perceived environmental constraints 
(e.g. funds available, student ability, administrative pressure, or national examinations) also will 
influence the decision, and their existence must be recognized. The presentation of evaluative 
information about the various physics texts will reduce the indecision of the teacher and increase 
the probability of an effective choice. 

In the curriculum area, there are several kinds of decisions to consider. Each decision 
type requires a specific evaluation focus to improve the probability of a successful decision. 

These types are shown in Figure 3. 


Fig. 3. Curricular decisions and evaluation. 

Type of decision Major decision-maker Evaluation required 
Improvement Course developer Formative 

Support of intervention Funding group Monitoring 
Utilization or recognition Course user Summative 


Formative evaluation addresses questions of course improvement during the development of 
à product. Monitoring evaluation is needed by the funding group (e.g. governments, publishers, 
or foundations) to answer the question of continuing support. Summative evaluation tries to 
provide information to potential course users (e.g. teachers) so they may make appropriate 
adoption decisions. 

In the remainder of this chapter, we will review the available evaluation information that 
addresses each of the three decision areas in integrated science. But first it will be necessary 
to provide a more specific meaning of integrated science education. 


Integrated science 
Integrated science education has been advocated as a 


ills. 
Science 


panacea for a multitude of science teaching 

It has been offered as a way to increase scientific literacy, understand the processes of 

Ios increase interest in Science, meet learner needs, maintain flexibility and show the 

relationship of science and Society. But these are the goals of nearly all science programmes 

san ied pias d defining integrated science. An evaluative look at integrated science 

ri tected quis its unique aspects in order to clarify how integrated science differs 
ience. ^ review of the literature! yielded only five characteristics which 


1 : 
Cohen, E Cit., T. Gadsden Jr., et al, " Cutting Boundaries with Correlated Science", 
School Science and Mathematics, Vol. 75, No. 1, p. 80-86; S.M. Haggis, "Attributes of 
Integrated Science Units", in Judith Reay (ed) 3 


s Final Report of Integrated Science and Teacher 
Education (ISTE) Workshop, St. Augustine, Trinidad: 


- University of West Indies, 1973; 
Showalter, "Rationale for ...", op. cit. ; and I.L. Slesnick, "The Effectiveness of a Unified 
Science in the High School Curriculum", Journal of Research in Science Teaching, Vol. 1, 

p. 302-314, 1963. 
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seemed to distinguish integrated science from other programmes. Most other claimed 
characteristics would apply to any well designed science course. Figure 4 is a listing of the 
unique characteristics of integrated science. 


Fig. 4. Characteristics of integrated science. 


Ls Traditional subject matter boundaries are phased out. 

2. The course usually lasts for two or more years and is sequential. 

3. The sequence tries to avoid duplication of content. 

4. The course usually serves a general education function (i.e. to develop scientific literacy). 
5. The course is organized around a selected unifying theme or aptos 


These characteristics were the criteria used for selecting evaluative studies of integrated 
Science in this chapter. These studies were examined to discover what information is currently 
available for the different decision-makers - developers, funding groups and teachers. 


Formative evaluation results 


Formative evaluation is usually carried out by course developers to assist in the decisions required 
for course improvement. Typical procedures include pilot testing materials in schools, reviews 
by teachers of preliminary versions of textbooks, and obtaining from scientists checks on the 
accuracy of the subject matter. Occasionally, achievement tests are used to determine if 
students are learning the intended content. 

Most formative evaluation results are not published because of their low generalizability. 
However, the case studies in the present volume provide some good examples of the procedures 
(and problems) involved in producing information for curriculum improvement. (See particularly 
the chapters on the Caribbean Integrated Science Projects by Reay; the Scottish Integrated Science 
Syllabus by Jeffrey; the Integrated Science Teaching in Malaysia by Sim; and the African Primary 
Science Programme by Yoloye). 

These chapters present several examples of how formative evaluation is usually conducted. 
Further examples of extensive formative evaluations have been reports by Welch! -and by Cohen?. 
Cohen's chapter in Volume II of this series is particularly illuminating on two extensive formative 
evaluation plans - the Australian Science Education Project (ASEP) and the Schools Council 
Integrated Science Project (SCISP). The ASEP project is one of the few published examples in 
integrated science where the formative evaluation has influenced revision decisions. No doubt 
there are others but these are not well documented. 

Whether or not the use of formative evaluation has improved an integrated science course 
is unknown. It seems apparent it has resulted in changes. It is likely these changes have 
been.improvements but no direct test of this fact was uncovered in the review of literature. 

The research on the effectiveness of formative evaluation as a means of improving decision-making 
is virtually non-existent. This conclusion is not limited to integrated science but is 
characteristic of formative evaluations in various areas. 


lw.w. Welch, "Curriculum Evaluation", Review of Educational Research, Vol. 39, p. 429-43, 1969. 
2 i 
Cohen, op. cit. 
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Monitoring evaluation results. 


No studies were found in the search of the published literature in the area of "monitoring 
evaluation". First, the term is not widely used even though it was proposed by Scriven! at the 
same time as his widely-used formative and summative terms. It refers to the generation of 
information for the supporting agency to reach decisions on the need for intervention and/or 
continued support. 

The case study by Sim describes quite well the need for such information in addition to 
providing a few examples of how this information was used in Malaysia. This is one of the few 
published examples of how evaluation can respond to the need for monitoring decisions. 

J am somewhat familiar with the monitoring efforts of the National Science Foundation, 

a large science funding agency in the United States. In general, very informal procedures are 
utilized - site visits, proposal review and required periodic reports. These are used for both 
integrated science and other courses. The monitoring procedures have come under recent attack 
because of potential bias or discrimination in funding decisions. Considerable effort is now being 
made to improve monitoring evaluation. The intent is to improve decisions on the intervention 
and support of curriculum projects. Of particular interest is the need for information to help 
decide if integrated science should be supported in a given country. Some assistance for this 
decision may be found in the summative evaluation studies reported next. 


Summative evaluation results. 


A substantial literature search was undertaken seeking summative studies of integrated science 
programmes. All issues of the United States published Science Education, School Science and 
Mathematics and the Journal of Research in Science Teaching were examined from 1970-1975. 

Any title found that might pertain to integrated science at the secondary level (ages 12-18) was 
reviewed and abstracted if it provided evaluation information. The review was limited to the 
secondary level because no effective way was found to separate integrated science from traditional 
science at the elementary level. A similar procedure was followed for the Australian-produced 
Science Education Research, issues 1971-1974, and Volumes 1 and 2 of Studies in Science Education 
from the University of Leeds. In addition, doctoral theses of the past twelve years were 

searched and six were found that had studied various integrated science programmes. 

The purpose of this search was to identify those summative evaluation studies which provided 
specific information on integrated science programmes. A further restriction was that programmes 
meet the definition of integrated science given previously. Surprisingly, our search yielded only 
thirteen studies and only seven of these met the tenets of disciplined inquiry; namely - systematic, 
objective and founded on a test of evidence. 

Three studies of programmes“ perceived by some writers as integrated science were 
examined. However, the programmes were of one-year duration only and, therefore, failed to 
satisfy the definition of integrated science used in this chapter. The Klopfer-McCann and Cossman 
studies were of one-year unified science courses offered at university laboratory schools, while 
the Gardner study was of an interdisciplinary chemistry course. The first two studies found 


1 


M. Scriven, "The Methodology of Evaluation" (1967), reprinted in Blaine R. Worthen and 


James R. Sanders (eds), Educational Evaluation: Theory and Practice, Worthington, Ohio, 
Charles A. Jones, Publishing Co., 1973. 


2 
G.W. Cossman, "The Effects of a Course in Science and Culture for Secondary School Students", 
Journal of Research in Science Teaching, Vol. 6, No. 3, p. 274-83, 1969; and L.E. Klopfer and 


D.C. McCann, "Evaluation in Unified Science: Measuring the Effectiveness of the Natural Science 


Course at the University of Chicago High School", Science Education, Vol. 53, p. 155-64, 1969; 
and J. Gardner, "The Interdisciplinary Approaches to Chemistry (IAC) Program and Related 


Research" in R.P. Tisher (ed), Science Education: Research, 1973, University of Queensland, 
Brisbane, Queensland: Australian Science Education Research Association, 1973. 
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several positive effects on measures of scientific literacy (e.g. Test on Understanding Science), 

but both studies were specially developed courses rather than multi-year integrated science curricula. 
Although I would be the first to admit that our search might not have included all papers, my 

initial reaction was one of surprise at the limited number of evaluation studies on such a popular 

movement. However, my goal was not number of studies but, rather, the important information 

contained in those studies. Unfortunately, my impressions did not change as the results of these 

studies were examined. 


Results. 

By far, the most popular target for evaluation studies has been the integration of chemistry and 
physics 1. Furthermore, a number of similar studies was reviewed extensively in Johnson which 
dated back to 1938. However, these studies are outside the time frame of the current review 


(1963-1975). It should be noted, however, that nothing in the earlier studies is contrary to the 
conclusions reached by the author for the more recent period. 

Although some may argue that the chemistry-physics integration is not unified science, it does 
meet the operational definition of integrated science presented in Figure 4. We are also fortunate 
to have several replications of a similar programme so that any conclusions reached are based on 
more than one integrated science sequence. 

Each of the six studies had slight variations in content and in criterion measures. For 
example, Johnson used the Test on Understanding Science (TOUS) as his measure of science process, 
while Fiasca used the Watson-Glaser Test of Critical Thinking. Similar variances existed on the 
achievement measures, although the Co-operative Science Test was the most popular. A summary 
of the six studies is shown in Table 2, over page. 


los. Bundy, Comparison of Achievement in an Integrated Two-Year Chemistry-Physics Course 
with Achievement in Chemistry and Physics taught as Separate Courses. (Unpublished doctoral 
thesis; Indiana University, Bloomington, Indiana), 1969; M.A. Fiasca, Feasibility of the 
Integration of Selected Aspects of (CBA) Chemistry, (CHEMS) Chemistry, and (PSSC) Physics 
into a Two-Year Physical Science Sequence. (Unpublished doctoral thesis; Oregon State 
University, Eugene, Oregon), 1966; F.D. Goar et al, "The Moline Project", The Science 
Teacher, Vol. 33, p. 40-41, 1966; J.D. Jacobs, The Development and Evaluation of a Unified 
Chemistry-Physics Course Taught by a Cooperative Teaching Method for a Two-Year Period. 
(Unpublished doctoral thesis; Pennsylvania State University, University Park, Pennsylvania), 
1967; G. Johnson, "An Integrated Two-Year Chemistry-Physics Course Compared with 
Consecutively Taught Separate Courses", Science Education, Vol. 59, No. 2, p. 143-54, 1972. 
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Table 2 reports the results of comparing the integrated physics-chemistry course with a 
two-year sequence of chemistry and physics taught separately. In all cases except Fiasca's 
study, the integrated course was locally developed and taught by the person conducting the study. 
All were two-year studies carried out at the upper secondary level (ages 15-18). 


Table 2. Evaluation results for integrated physies-chemistry courses significant differences. 
Study Criterion tests 
Chemistry Physics Physics-Chemistry Process Student 
achievement achievement achievement of interest 
Jacobs 0 0 0 0 
Bundy - = = 
Johnson* + 0 = 0 + 
Lerner 0 0 
Goar °° 0 0 
Fiasca 0 0 9 0 


Go ONERE CAL 


Key: * integrated science higher 
0 no difference 
- integrated science lower 


* Because I seriously questioned the appropriateness of the adjustment procedures, only 
unadjusted comparisons are reported here. The original study reports both. 
*' Statistical tests not reported. ^ Percentile differences reported are not significant. 
A total of twenty comparisons were made in the six studies. Two comparisons favored the 
integrated course, four showed the separate sciences higher, and fourteen resulted in no 
significant differences. The implications are clear. There is virtually no evaluation evidence 


in these studies to support integration of physics and chemistry over the teaching of the courses 
separately. It appears to make very little difference using the criteria of achievement, 
understanding the processes of science or student interest in science. 


This conclusion is in sharp contrast with evaluation interpretations reported elsewlere 
In order to resolve this apparent difference of opinion, let us examine the other integrated science 
studies found in the literature search. Two studies? of the unified science programme at the 
now-defunct University School at Ohio State appear to have been particularly influential in 
providing evaluation evidence in support of integrated science. It is my contention that both 
studies have sufficient problems to warrant equivocal conclusions consistent with the integrated 
chemistry-physics findings. ^ The Showalter study will be considered first because it is more 
recent and addresses a broader range of evaluation criteria. One-hundred-and-eight graduates 
of a unified science course from 1963-1965 were compared with one-hundred-and-eight 1960-1962 


1 
Cohen, op. cit., Showalter, "Rationale for ...", op. cit. 


2 
V.M. Showalter, Effects of a Unified Science Curriculum on High School Graduates. 


(Unpublished doctoral thesis: The Ohio State University, Columbus, Ohio), 1968, Slesnick, op. cit 
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graduates of a traditional sequence at the same school. These are labelled "Experimental a" 


and " Control a" in Table 3. A further comparison was made between seventy-one matched pairs 
of the 1963-1965 graduates at Ohio State and similar graduates at a traditional Ohio school 
(called "Experimental b" and "Control b"). The groups were compared on a variety of measures 


including college grades, science interest, science literacy test, and questionnaire items. 
A summary of the findings is found in Table 3. 


Table 3. Unified science versus traditional science** Test means. 
Groups Criterion measures 
Science Years Hours of s College Five 
$ Science " College " . 
literacy of College interest preparation ades questionnaire 
test Science Science perception g items 
imental a 29.7 3. 59* 7.18 12.3 19.01 2.22 
Pappsim No differences 
Control a 27.4*7 2.77 6.06 11.3 17.42 1.92 
E rimental b 29.2 3. 76* 7.02 12.7 19.26 2.29 
di No differences 
Control b 31.4 3.20 7.31 11.8 18.08 2.46 


MM — — MM aa 


* Indicates a statistically significant difference. 
Z Low score indicates higher science literacy. 


** From Showalter (1968). 


A total of twenty-two separate comparisons were made. Only two of them (years of 
Secondary school science) favored the unified science group when statistical tests were applied. 
The control group was significantly higher only on the Science Literacy Test developed especially 
for this study. The remaining nineteen comparisons were not different. Thus, I believe the 
only justifiable conclusion is the one reached earlier. There is no evaluation evidence in this 
study to support the claims that integrated science achieves the objectives of science literacy 
better than the separate science approach. One apparent conclusion is that students who sign up 
for a four-year sequential course tend to stay in that sequence more than do those students who 
must decide upon a new science course each year (see column two). 

The Slesnick study was an evaluation of the first three years of the Ohio State University 
School programme described above. The single criterion instrument was a measure of an 
individual's conception of the physical world using a test of a "rational image of the universe", 

A matched group was chosen from another school and seventy-eight matched pairs became the 
subjects of the study. The major finding was that the unified science students scored higher 

on this test than did the control group. Although there are methodological errors in the report 
(e.g. Table 4 on page 311 use of t-tests to test proportions and incorrect designation of sample 
size), my major concern is an assessment of the programme using a test developed by the same 
person who taught the experimental group. It also appears the test is content-specific; 

that is, it was designed particularly for the unique content of the unified course. This is hardly 
an unbiased measure of course outcomes. 

These concerns do not imply that the results are incorrect but, rather, they must be viewed 
with extreme caution. Further verification is needed. Little was found in the remaining five 


studies to contradict this conclusion. 
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Hall t reported on an evaluation of SCISP developed for 13-16 year old pupils. He reports 
there were significant differences between the means of SCISP and non-SCISP pupils of the same age. 
However, he does not report which group scored higher. Furthermore, other aspects of the study 
are not clear, e.g. control group characteristics , statistical tests used and group means. 

Several of the studies reported the positive reactions of the integrated science experiences as 
perceived by the teachers involved in teaching the programme. In many instances, evaluators 
reported that the programme seemed to work well but did not provide supporting data. The one 
exception to this general rule was in the area of student enrolments. Herr”, Pfeiffer 3, Fiasca 
and Gadsden all report increased science enrolments after the implementation of an integrated 
science curriculum. 

Whether or not this represents a true expression of student reaction to the movement, or is 
merely an artefact of sequencing Or the "halo" effect, is unknown. Information on this issue is 
currently not available. Certainly, it is an area which requires further study. 


Conclusions and Recommendations 


The purpose of this chapter has been to examine the information available to assist developers, 
funding groups and teachers in making decisions about integrated science. Evaluation and 
decision-making were defined and several essential characteristics of the integrated science 
movement were identified. A search of the literature of the past dozen years revealed few 
studies bearing on the problem. In the opinion of the author, what is available provides very 
little evidence to answer either utilization or support decisions. Furthermore, the value is 
unknown of the formative evaluation which has been conducted. Descriptions of projects are 
available and the number of integrated science programme throughout the world suggests a great 
interest in the movement. Unfortunately, the situation is little changed from 1969 when this 
reviewer examined 204 documents in science education. At that time I wrote, "a finding 
surprising to this reviewer was the small number of studies devoted to unified science. Recent 
activity suggests a movement in this direction. The fact that only four studies reported during 
a two-year period (1968-1969) were concerned with this topic indicated little research interest in 
the movement 4". (Welch, 1972, p. 98). A similar situation appears to exist today. 

Given the lack of valid and credible information and the need for some key decisions 
regarding integrated science, I would strongly urge the planning and initiation of several well 
designed evaluations of the more popular programmes in existence today. Longitudinal studies, 
perhaps funded by international agencies or national agencies such as the Nuffield Foundation or 
the National Science Foundation and carried out by impartial evaluators, are sorely needed. 
Philosophically and psychologically, a strong case can be made for the place of integrated science 
in our curriculum. A valid test of this rationale is needed by teachers, the public, and the 


funding agencies. Information is required to make effective and efficient decisions regarding the 
role of integrated science in our schools. 


1 
WC. Hall, "A Description of the Development of Assessment Procedures for the Schools Council 
Integrated Science Project", in Fisher, op. cit. 


2 


L.G. Herr, "Unified Science: A solution to Physics Enrolment", The Physics Teacher, Vol. 9, 
p. 248-52, 1971. 


3 * 
Gei Pfeifrer, The Development and Implementation of a Four-Year Unified, Concept-Centered 
Science Curriculum for Secondary Schools, Final Report. ERIC Document ED 054 965 


SE 021 468. (ERIC Document Reproduction Service, P.O. Box 190 Arlington, Virginia 22210, 
U.S.A.), 1969. 


4 " 
W.W. Welch, "Review of Research 1968-69 in Secondary Level Science", Journal of Research in 
Science Teaching, Vol. 9, No. 2, p. 97-122, 1972, p. 98. 
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SUMMARY 


Three functions of the analysis of curriculum materials are explored: training, research and 
evaluation. Emphasis is placed on the latter in relation to the goals, the setting and the role 
of the evaluator. Formalized schemes for analysis, created in the United States, the United 
Kingdom and the Federal Republic of Germany, are presented and reviewed in terms of the 
criteria for their construction and their use in analysing integrated science curricula. The 
concluding section of the paper considers the relationship of analysis and analysis schemes to 
integrated curricula in respect to both content and methodological considerations. 


The analysis of curriculum materials 
The functions of an analysis 


The analysis of completed curricula materials, either student materials or teacher materials or 
both, using one or another of the formal schemes described below, can have several functions. 
The analysis itself can have a learning function. A teacher or a student teacher can learn 
much about the components and structure of a curriculum by analysing it. The analysis of a 
curriculum, or parts of several, should make pre-service or in-service discussions about 
curricula more realistic, as well as providing detailed information about curricula analysed. 
By using a formal scheme as a guideline for analysis, much information can be gained. Such 
pre-structured schemes limit subjective critiques or emphases on one area or another by 
Separate analysts. Also, however, such a scheme necessitates that the person using it is 
trained in its use, a beneficial pre- or in-service situation. 
An analysis of curriculum materials can also service a research function. The authors of 
the Sussex scheme designated "this as a curriculum criticism" function: 


n ... we prefer to use the term 'curriculum criticism' to describe a curriculum analysis 
which is not specifically decision-oriented; and see the main purpose of such criticism 
as the disclosure of meaning and the extension of knowledge about the curriculum. The 
critic, unlike the evaluator, is free to choose his own standards and values and to focus 
on particular issues rather than attempt to cover a wide range. Moreover, it is he, 
together with other educational researchers, who should provide the basic knowledge on 
which the evaluator can draw. The evaluator is essentially a technologist and his 
service role depends on the state of educational knowledge". 


lw. Eraut, et al, The Analysis of Curriculum Materials, Brighton, Sussex, University of Sussex, 
Occasion Paper 2, 1975. 
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Here the emphasis could be on extending knowledge for curriculum developers, for example, 
on what is available and what it is like with regard to structure, content and integrating principles. 

An analysis of curriculum materials can also have an evaluation function. Here we must 
very carefully set out what is meant by evaluation and for whom. There comes a time when a 
teacher or a supervisor or an administrator wants to recommend for usage a curriculum which 
suits his own needs, attitudes, background or whatever. Such recommendations should be justified 
against "acceptable" or "desired" criteria. 

The evaluator engaged in an external analysis of curriculum materials is placed in the position 
of decision-making. This he can do principally on two levels: by orienting his interest to 
internal or external relevance. Internal relevance indicates how the set goals are met by the 
concrete curriculum documents or how intracurricular, axiomatic agreed-upon requirements fit 
with the programme in question. It comprises an estimation of how the documentary and 
empirical evidence is consistent with respect to supra-ordinate goals and theoretical statements 
within the document or how representative the data are for a given concept. Internal relevance 
is akin to evaluating an industrial product with respect to its initial specifications or to generally 
accepted standards. Quality control and assessment of the product and its performance remain 
within the scope of agreed-upon levels, factual needs and accepted specifications. 

This pre-supposes a given value-system, within which refinements are related to the 
accepted standard goal. The assessment of how well the final product meets the initial aim 
Operates on the operationalized level of both specifications and data: reproducible, quantifiable , 
describable, unambiguous. 

The internal relevance approach is valuable for documentation purposes and serves as a 
basis for information. It suffers at the same time the danger of triviality, as Smith warned: 


"It seems ... that a remarkably high proportion of the research reported is clean, 
stringently conceived, and effectively executed, reflective of rigorous and painstaking 
thought and experimentation, and remarkably trivial! There are occasions when I 
have the unpleasant fantasy that psychology has become so enamored of method that 
techniques become our independent variables and our substantive problems only the 
dependent ones", 1 


External relevance indicates how the given document and its implicit goals relate to external 
requirements. This implies: (1) sensitivity towards trends and changes within the value system, 
(2) anticipation and sketching in outline the features of foreseeable needs and goals; and (3) the 
evaluation of the theoretical frame of a curriculum and the evaluation of its importance. External 
relevance seeks to determine the discrepancies between the curriculum goals and the external: 
requirements. The criteria beyond the scope of the curriculum system can be derived from 
theoretical deductions of societal requirements, the research interest of the evaluator, the 
reflected intentions towards a different society and the factual evidence of disagreement and 
discrepance between the theoretical system and the actual requirements. 

By analogy to industrial products, in the case of external relevance, the product will not be 
subjected to quality and performance control with respect to the initial specifications, but the 
Specifications will, first of all, be the subject of an evaluation of their relation to changed and 
changing value systems. After this kind of assessment, the feasibility of internal relevance of 
a curriculum gains importance. 

Decisions based upon internal relevance use criteria often restricted to a descriptive 
analysis of the documentary evidence of the curriculum materials or empirical evidence from 


responses to the curriculum. The evaluator might claim neutrality by working within the system, 
but this is debatable. 


1 . signee 
M.B. Smith, Editorial, Journal of Abnormal Social Psychology, No. 63, 1961, p. 461-65. 
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In the introduction to the Sussex-scheme The Analysis of Curriculum Materials, the authors 
enter the debate about the evaluator's role as being more than simply about that of description. 1 

They contrast Scriven's assertion that the evaluator must judge, that curriculum evaluation 
should involve comparison between alternatives and lead to definite conclusions, with other 
viewpoints which maintain that the evaluator should stop short of passing final judgements. The 
basis of this view is that different people in different contexts have different standards and different 
values, The Sussex authors contend that the evaluation should respect these values and provide 
evidence (documentary or empirical) for helping decision-making groups to make their own decisions 
on their own terms. 

Goal- and value-guided research enters the field of doubt: for instance, are the priorities 
Set for certain objectives sought by the evaluator based on personal preference or on a valid 
“reasoning which reflects "objective" and "true" need? Are the criteria for judgement of extrinsic 
elements (that is, those elements that do not relate to intracurricular features such as 
consistency) gained by a constant process of "responsive evaluation" (Stake), by general consensus 
Of a general community, or do the criteria stem from a theoretical analysis of the society and a 
prognostic synthesis of a desired society? It can be stated that these value concepts are not 
problems of "personal preference", that a common denominator which reflects intersubjective 
validity exists. It is well known that the assessment of judgement data is difficult. Scriven 
claims: ^"... value judgements are one of the slipperiest species in the whole logical zoo" 2, 
while Stake describes judgement data as messy 3. 

The task of assessment is related to the heterogeneity of values. Stake remarked that 
"the evaluator's task is to reduce the apparent heterogeneity to a manageable representation ... 
in a simple yet valid way".4 This recognizes the priority of value judgements, the secondary 
Position of the methodology, and the provision and analytical treatment of judgement data 
(see also the discussion of decision-making by Welch in the previous chapter). The role of the 
judgement data within the process of decision-making might be viewed as providing guidelines 
for the evaluator to reach decisions between var ous curriculum programmes or in support of the 
construction of a new programme to meet his standards. The study takes its direction from the 
investigators commitments, belief, political position or, in many important cases, hunches. d 

Intentionality of the evaluator can be seen as an important component. The context of 
intentionality operates within the anticipation of not-yet-established goals, within the concept of 
"from Is to Ought".  Intentionality also includes stated external criteria which relate to a value 
System but, at the same time takes advantage of data provided by descriptive analysis along with 
the methodology of assessing these data. The role of intentionality is one of the central themes 
within curriculum analysis systems and plays a key role in the description-evaluation debate. 

This problem of "two cultures" is also recognized by the authors of the Sussex Scheme: 


"Analysing materials is an obvious and convenient form of educational enquiry; and we 
expected to find considerable support in the literature. However, whenever we looked 

for guidance, whether it was to philosophy, psychology, to curriculum theory, to sociology, 
or even to practical books on the methodology of particular subjects, we seemed to find 

a large gap between what was written and what was needed for our analysis. We found 
plenty of theorising about what ought to be done and plenty of practical advice on what to do: 


1 r 
Eraut et al., op. cit., p. 12. 
2M Scriven, Value Claims in the Social Sciences, Boulder, Colorado: Social Science Education 


Consortium, Publication No. 123, 1966, p. 1. 


ŽR.E. Stake, Objectives, Priorities, and other Judgement Data. Review of Educational Research 
(Boulder, Colorado), Vol. 40, No. 2, 1970, p. 182. 


^ ibid., p. 184. 
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but there were no links between the two and there was little or no critical examination of 
the assumptions underlying practice. We had to conclude that, with a few notable exceptions, 
publications that might be appropriately labelled 'curriculum criticism' do not exist". 


Their work in curriculum analysis has led the Sussex analysts to believe that "many decisions 
in the field of curriculum are based on intuitively held unformalized knowledge", therefore, . 
"the importance of curriculum criticism is hard to deny. It is needed both to illuminate existing 
knowledge and as a springboard for the heuristic leaps of the future". 


Curriculum Analysis Schemes 


In this section, three of the available analysis schemes will be examined, together with the criteria 
to which they do, or do not, attend. 


The Curriculum Materials Analysis System (CMAS), developed by the Social Science Education 


Consortium (SSEC), in Boulder, Colorado, was one of the first and in its longer forms, is still 
one of the most comprehensive of the analysis schemes. It exists in three versions - short, 
intermediate and long - each intended to have different purposes. The long form, offering a 
complete, in-depth analysis, is intended for use by students and professors in methods courses 
at university. The intermediate form is less detailed and offers a format more useful to 
curriculum committees and for in-service training. The short version is designed to give a 
brief overview of the curriculum to aid curriculum committees, supervisors, etc., in acquiring 
i i i i lection. i 
parria ps n a the CMAS was the increase in number of available curricula 
in the social sciences in the United States in the 1960s. The creators of the cepe qne dm 
developed the scheme and carried out analyses of various social studies Por ap ed cs s 
in the production of several reports, called CMA's which stands for Curriculum E ri i mà E 
(Both the scheme in its three forms and the various pne P other supporting docume 
available from the SSEC headquarters in Boulder, Colorado. . 

i dat ie of the scheme (SSEC, 1971) will be used here as a basis for 
further clarification. This version has eight sections. They are: 


1.0 Product characteristics. 

0 Rationale and objectives. 

0 Content. 

0 Theory and strategies. 

+0 Antecedent conditions. 

0 Evaluation. 

0 Background of materials development. 
0 Background of the analysis. 


Each section is divided into two to six headings; each heading may be further divided into as 
many as six sub-headings. For example, section "3.0 Content" is divided into: 


3.1 Cognitive content. 
3.11 Author's view of subject. 


3.12 Cognitive content of curriculum materials. 
3.2 Affective content. 


3.21 Author's view of affective content. 
3.22 Affective content in the curriculum materials. 


1 praut et al, op. cit., p. 14. 


?ipid., p. 15. 
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The section begins with a short clarification of what is meant by both cognitive and affective content 
The material also contains definitions of such terms as "fact", "concept", "value", and "attitude" i 
in order to ease the task of the analysts. For each of the headings and sub-headings there is a 
number of questions: for example, there are eight questions for heading 3.1, two düstiong for 
Sub-heading 3.11 and six for sub-heading 3.12. In total, there are twenty-four questions to be 
answered in section 3. Both the curriculum materials and supporting documents must be surveyed 
in order to answer all the questions. : 

The questions are of three general types: 


1. Most of the questions are of a semantic differential type, for example 3.1, question 7: 
®Do the materials emphasize the affective or cognitive content?" 


fu ds fopade fe fe 7 


A great deal of A balance A great deal of 
affective content cognitive content 


2. A few questions are of a check list type, for example 3.1, question 2: 
"What discipline(s) is(are) emphasized in the materials?" 


Anthropology 


Economics 
Geography 
History 


Political Science 


Psychology 


Sociology 
Social Psychology 


Interdisciplinary 


Multidisciplinary 


3. There are occasional open answer type questions, for example 3.1, question 3: 


"What other subject areas are emphasized?" 


The total intermediate form has 376 separate questions to be answered. No estimate is given of 


the approximate completion time. 
The analysis of the system is by no means complete, but nevertheless we can make 


Observations about the scheme. First, it must be pointed out that the scheme contains certain 
definitions, for instance, of "values" (a value is assessed worth toward a thing, event, behaviour 
Or phenomenon, 3.2, question 2), which are to be used by the analyst during the analysis. 
Assuming that these definitions are neither too "conservative" nor too "radical", an analyst should 
be able to maintain the scheme's stance during the work. Of the schemes presented here, this 
Scheme is unique in that it establishes certain criteria, that is, meanings for terms. 

The constructors have selected definitions which will cause as little disagreement as possible, 
whereas most other scheme constructors have left the question of criteria to the audience/analyst 


and thus maintained a neutral stance within the scheme itself. 
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The SSEC scheme was not developed with the natural sciences in mind. However, 


it is 
readily adaptable to almost any subject field, with the advantage that its emphasis in the areas of 
values and the affective domain is quite relevant. Should such a content section be developed for 


the natural sciences, it would be preferable not to use the semantic differential type question. 

By far the largest difficulty with the SSEC scheme is its complexity. The constructors 
seem to have realized this themselves, in that they set out to do analyses of various curricula 
using the long form. Even the intermediate form, for one not carefully trained in the subject 
matter and in the use of the scheme, is difficult and time-consuming. Unfortunately, the short 
form is considered by many to be too short to be useful. 

The SSEC, in its work in constructing the three versions with different audiences in mind 
and in its function as a clearing house for information about other such evaluation schemes in the 
social sciences area (they list some 800 schemes with different focus points) has promoted the 
notion of evaluation of curriculum materials. The SSEC scheme itself is a good example to those 
concerned in evaluation of science curriculum materials. It promotes the notion that adaptation 
of existing schemes rather than creation of others might be the way to approach the problem. 


A second analysis scheme, The Sussex scheme for the analysis of curriculum materials, developed 
at the University of Sussex, is not subject area dependent. Three advantages are claimed for 
usage of this formalized scheme for analysis of curriculum materials, namely: (1) provided the 
analyst understands the scheme, most of the major areas are covered; (2) it is useful when two 
or more competing materials have been analysed with the same scheme; and (3) the scheme 
provides a structure for the analyst's arguments and clarifies the assumptions and evidence base. 

Some general disadvantages of using such a scheme include: (1) the scheme must be one 
that any analyst can "make his own" without feeling either too free or too structured; (2) the 
danger of "overkill", that is, of spending too much time and effort in the analysis, of being 
sidetracked from the over-all goal by too much attention to detail; and (3) the communication 
problem, i.e. in what way and with what detail the report of the analysis should be disseminated. 

The authors of the Sussex scheme describe three stages within the total evaluation of the 
curriculum at which the analysis component has different roles and different audiences, but with 
the same goal, namely, "to analyse all the available evidence and relate it to different 
educational perspectives". 1 (See Table 4.) 


Table 4. The three stages of curriculum evaluation. 
Time Audience Purpose 
1. Formative evaluation Development team To guide further work 


2. Initial stage of 
summative evaluation 


Evaluator herself /himself To guide the subsequent 
stages of the evaluation 


3. Final stage of 


$ Decision-makers To guide their decisions 
summative evaluation 


Tne authors suggest that during the formative stage, the analysis of the early written materials 
of a curriculum development project should be analysed in terms of major assumptions about 
feasibility, desirability and educational value, as well as goal analysis, consistency analysis and 
contingency analysis. They suggest that content analysis might be beneficial and that a 
cost/benefit analysis, although difficult at this stage, might yield sufficient important information. 


1 
Eraut et al, op. cit., p. 13. 
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At the second stage - the initial stage of summative evaluation - the analysis is seen as 
having the function of hypothesis-formation for the evaluator as she/he guides and/or plans 
subsequent empirical investigations. ^ Without such an analysis at this stage (pointing out possible 
incongruencies between the intended and observed antecedents, transactions and outcomes), the 
authors suggest that data-collection by the evaluators could be misdirected. 

In the third stage - the final stage of summative evaluation - the analysis has as its role the 
combining of both documentary and empirical evidence into a concise final report which aims to 
inform decision-makers about the adoption or implementation of the curriculum. 

The authors of the Sussex scheme see the functions of such a scheme falling into several 
categories, not just evaluation. Three of these functions are described below: 


1. Curriculum criticism function: here the analysis is not specifically decision-oriented, but 
fa GAS as WES E ARE ERR 5 M 
rather has, as its main purpose, the "disclosure of meaning and the extension of knowledge 


about the curriculum".1 


2. Pre-service education function: depending upon where such an analysis scheme is used in 
pre-service education, its function could be quite different. The authors suggest that the 
use of such a scheme in a professional course to compare and contrast two rival curricula, 
or in a philosophy course to emphasize contingency analysis, can be beneficial. In this 
sense, a curriculum analysis approach can have an integrative function in pre-service 


education, but without careful planning a fragmentation can occur which would not be 


positive. 


the Sussex scheme arose when the authors were active in training 
teachers and others for curriculum work at the local level. Their experience suggested 
that an emphasis upon curriculum analysis has been successful in imparting the notions 
about, for example, what a curriculum is. Likewise, the authors outline three goals for 
short-term (one week) in-service curriculum analysis workshops for which they claim 
success. These are: to improve the implementation of new curriculum materials, to 
improve existing curricula and to guide the selection of curriculum materials. 


3. In-service education: 


The Sussex scheme itself is divided into four required and one optional part. The four required 
Parts are: (1) introduction, (2) description and analysis of materials, (3) the materials in use, 
and (4) evaluation. Part 5 (Decision-making in a specific context) is optional because it focusses 
On constraints, patterns of use and implementation of strategies for a particular situation. 


Part 1 (Introduction) has three major divisions: 


1.1 Basic facts. 
1.2 Author's rationale. 
1.3 Issues and perspectives. 


Section 1.1 involves a description of the title, author, publisher and cost of each ot the parts of 
the curriculum, a statement of the aims and functions of the material, a description of the 
audience (age "sahocl type, course duration, etc.), information about field testing (if any) of the 
Curriculum and additional information about the authors! credentials, background, etc. Both the 
curriculum material and supplemental materials must be used to complete this part of the analysis. 


l Eraut et al, op. cit., p. 23- 
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Each of the five parts is subdivided in a similar way, with specific areas mentioned for 
comment. For example, in Part 3 (The materials in use) one finds the questions: 


3.4.1 How much teacher time is needed prior to implementation for activities such as gaining 
familiarity with the curriculum, further planning, and selecting or producing further 
materials? 


A question from Part 4 (Evaluation) is as follows: 
4.1.3 What information about the users of the resource and their experience is available? 


The scheme is both structured and flexible, in that the questions are prescribed and yet do 
not limit inclusion of further areas of analysis which might be relevant for a particular curriculum. 
Likewise, the scheme is flexible in that there is no set mode of answering. (For a further 
discussion of flexible evaluation schemes, see Tawney's discussion in Chapter 1.) This, of course, 
limits the kind of reliability investigations that one can make about the scheme itself. The authors 
stress that training in the use of the scheme in one-week workshops is useful, if not necessary. 
This could decrease the usefulness of the scheme because of the lack of training personnel for 
such a workshop and because of the lack of attractiveness of such an activity for teachers and 
supervisors. It is estimated that a thorough analysis using all secondary materials, especially 
for Part 4 (Evaluation), would require one to two weeks of full-time work. Outside of activities 
in pre- or in-service training, this time seems long for the experience and/or information gained. 

Because the Sussex scheme was intentionally developed to be applicable to the analysis of 
any subject area curriculum for any age level, it lacks certain characteristics (for instance, any 
mention of integrating principles) which would make it more relevant to the evaluation of integrated 
science curricular materials. However, the bases on which the scheme was developed and author 
considerations of the functions and advantages /disadvantages of such a scheme make its consideration 
worthwhile. 


A third formalized scheme is the Curriculum Material Analysis Scheme for_Seience (CHES) 
developed at the IPN in Kiel, the Federal Republic of Germany. This scheme, as its title implies, 
was constructed specifically so that curricula in the natural sciences could be analysed. The 
authors contend that there are two groups which could benefit from systematic analysis of 
curriculum materials: curriculum constructors and researchers in curriculum design (to these 
groups might well be added decision-makers responsible for adoption of curricula). 

The five points of emphasis in the construction of this scheme were: 


Adaptability: it should be possible to later add questions not formally considered during the 
design of the instrument without changing the pre-existing structure of the instrument. 


2. Specificity: information should be provided on more than one level. 


Reliability: the use of the instrument by separate analysts on the same material should lead 
to similar results. 


4. Quick access to the data: 


; the data compiled should be quickly, succinctly and continually 
available. 


5. Ease of use: little or no training beyond an "Instruction for the user" manual should be 
needed in order to use the system reliably. 


i Various siraciural features of the system have been incorporated to meet these five 
requirements. For instance, each of the various categories of analysis developed stands by itself 
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and gives its own information. However, by differentially combining the categories, other questions 
can be asked and answered. The use of a punch-card system for data storage and retrieval helps 
fulfil the requirements for adaptability, quick access and ease of use. 


Table 5. Five major areas with one or more prime components. 

Major area Prime components 

aN ——$—$—$—$—$——— 

Content Behaviourial elements; subject matter elements; general elements. 

Instructional methods Management; learning styles; educational technology and media. 

Adaptiveness Individualization; learning rate; integration. 

Effectiveness Evaluation techniques used in the curriculum; evaluation studies on the 
curriculum. 

Administration Cost; implementation; teacher training. 


o ———————————————————————————————— 


Because the CMAS was especially developed for analysis of science curricula materials, a 
prime area of interest is the subject matter analysis. The designers of the CMAS had eight 
requirements in mind when they constructed this part of the scheme, related to such factors as 
differential level of specificity, approach, comprehensibility, flexibility and cross-science 
references. These considerations led to the construction of four categories to indicate content. 


They are: 


Column 45; Major area of subject matter 


; 4 = astronomy 
0 = 
biology 5 = mathematics 
1 = chemistry c= eneral to sci 
2 = physics : —: 
3 = geology or meteorology 


Column 46: General approach to subject matter 


i 5 = economical 
1 3 a pedcs Gr te technological 
2 = chemical gy om historical or biographical 
3 = physical 8 = mathematical 
9 = environmental 
4 = sociological 


Columns 47-49: Specific ke ords 


lists, a general alphabetical list and a subject-oriented list 


F is indi d in two 
Each keyword is indicate geology, astronomy, mathematics and general to science). 


(for biology, chemistry, physics, 


Column 50: General keyword 


3 3 = structure of 
A = mone of the BST 4 = application or use of 
= measurement 0) 5 = instruments in the field of 
2 = forms of 
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Using these six columns, one can construct sentences describing a section of the curriculum as to 
content. For instance, if Column 45 = biology, Column 46 = chemical, Columns 47-49 = DNA 
and Column 50 = structure of, the description of the content could read: 
to the structure of DNA". 

The use of columns 45 and 46 help in a limited fashion to describe whatever integration there 
might be of content in the curriculum. Likewise, emphasis in Section II of the CMAS concerning 
the total curriculum (not individual sections) is placed on analysing the principles used to 
"co-ordinate or integrate various science disciplines". 

The CMAS is one of the few schemes whose reliability has been tested. The constructors 
report a study in which fifty of the categories were analysed. In forty-seven of these fifty 
categories the inter-analyst consistency was acceptable at the 5 per cent level. The three 
categories where the consistency was not acceptable were in the knowledge categories, where 
distinctions are not clear to a non-specialist. 

The CMAS is structured so that one does indeed learn a great deal about particular 
randomly-chosen parts of a curriculum (in the Section I analysis) and an overview of the total 
curriculum (in the Section II analysis). Perhaps one feels more secure about using this system 
in the analysis of science materials because it was developed especially to do that. However, 
outside of the subject matter analysis, it may well be that one of the more general schemes 
could offer more comprehensive information. about the total curriculum process. What none of 
these schemes, nor any other known scheme does, is allow us to evaluate, or even analyse, 


the methods or extent of integration in science curriculum materials. 


"a biochemical approach 


The relationship of analysis-evaluation schemes to selected bases for the integration of science. 


The nature and the analysis of integrated science curriculun., materials can be linked in a 
productive inter-relationship in which the analysis scheme stretches the curriculum criticism 
function towards actual curriculum construction. 

If we picture the situation in which the analyst faces the question: "Does a curriculum . 
exist that meets my criteria?" , a first systematic search might consist of seeking an analysis 
Scheme that matches the criteria. This is the first confrontation of initial value judgements with 
the schemes. If a scheme is not located, the list of criteria might be used to develop an 
evaluation scheme based upon a set of stated requirements to be met by an instructional programme 

These requirements would not derive from common practice or traditional intentions within 
curriculum construction, but from deductions, for instance from the field of social theory, 
philosophy or philosophy of science and would be related to the objective within the concept of 
what has not yet been and needs to be established. This analysis scheme can contain elements 
which are not sufficiently or at all represented in existing science curricula with an integrated 
approach. The scheme gains, then, the normative status of postulating the need for these 
elements and can thus serve as the backbone for a curriculum to be constructed. 

The second basic question of the analyst could be: "What are curricula like that call 
themselves integrated?". ^ Here the focus is on the various principles of integration; the scheme 
Should be able to speak to these principles. 

The combination of both questions produces a situation in which both the curricula of an 
integrated nature and the schemes that could be specific to integration are questionable. Therefore 
the concluding statement of the previous section reads: "What none of these schemes, nor any 
other existing scheme does, is allow us to evaluate, or even analyse, the methods or extent of 
integration in science curricular materials"; that is, as has been stated before, the schemes 
cannot consider integration more than the actual documents do if the categories of the analysis 


schemes have been constructed from the documentary on empirical evidence of the curricula 
claiming a unifying or integrating approach. 


P. Haussler and J. Pittman, A Curriculum Material Analysis System for Science, Weinhem and 
Basel: Beltz Verlag, 1973, p. 179. 
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The schemes can be, therefore, considered as analysis instruments for "normal" curricula and 
not specific to integration. An analysis system such as the CMAS of the IPN, Kiel, provides a 
basis for surface evaluation without telling much (if anything) about integration. If such a scheme 
were to analyse the integration aspects of an integrated curriculum, it would have to speak both to 
the content aspect and to the method aspect. 

Questions as to the content aspect are: (a) what is integrated?, and (b) what are the principles 
used to integrate? 

Question (a) contains a reference to the problem as to what can and should be integrated. 
Basically and traditionally, one of the main underlying concepts of science curriculum projects of 
an integrated nature is the assumption of a unified universel and the basic unity of science 2. 

The notion of unity has, of course, an older history. William James, for example, 
criticized in the last century the atomistic view point which attempts to 


"chop up the mind into distinct units of composition or function, numbering these off, and 
labelling them by technical names". 3 


Additionally, the view of a unitary whole, represented in the writings of John Dewey among 
others, supplements this concept development. 

As for science, emphasis on the unity of the universe was made by the developments of the 
Wiener Kreis of the late Twenties - the neopositivistic circle around M. Schlick in Vienna, later 
represented by R. Carnap, H. Reichenbach and H. Feigl. Indeed, "unity of science" is the exact 
translation of Einheitswissenschaft, the term which was used synonymously for neopositivism: the 
journal of the Vienna Circle, Erkenntnis, continued as Journal of Unified Science after 1938. 

Under the assumption of the "natural world in a piece", of course, nearly anything can be 
undertaken in order to restructure and reorganize science for instructional purpose. The unity 
pool thus provides the basis and elements for organizing principles which relate to the unified 
nature of science. The notion of unified science or the attempts to integrate attains thus the 
quality of operations within set theory where given elements can be organized according to sets 


of intended quality not, however, the not-yet given or recognized elements. 


The problem now arising within the content area for integration concerns the validity of 


integrating within the sciences. 
in its social context is not spoken to 
disciplines as history or politics? — 
analysis scheme or a curriculum wi 


NCC reasingly important field of science-related social issues and society-related science 
e inc 


i tion projects of an integrated nature, with very 
i by any of the science educa 
A inate sme si as the curriculum Man and the Environment, parts of the IPN 
1 


i 1 ary education, some modul e Ope University. 
Curriculum Physik and, for tertiary , es of The Open i 
As therefore hardly any evidence egrated approach overlapping and extending 


ah ti-disciplinary content incorporating the 
i i establishing a truly mul 
eidem ben ye Tape and society, the analysis schemes have not accounted for 


integration of this nature. 


Can one, therefore, legitimately talk of integration if science 
, if the linkage models do not cover such "external" 

The evaluator seeking a differential answer by means of an 
ll not find himself in a position in which his needs will be 


1 " 
Rutherford and Gardner, op. cit., 47 ff. 


2v M. Showalter, The Unity of Science. In; What is Unified Science Education? Columbus, 
M. OW: x . 
Ohio: Centre for Unified Science Education, 1974, p. 11. 


31 T. Hopkins, Integration: Its Meaning and Application. New York: D. Appleton-Century 
ot. pkins , H 


Company, 1937. 
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Question (b) relates not only to the principles, but also to the value of these principles. 

What is indeed gained by reconstructing and reorganizing science content by operating from the 
"unified pool" only and seeing science education as a matter of set theory? 

The function of a scheme should be to evaluate the value of the structuring principles of a 
Science programme with an integrative nature. The value of a conceptual framework for cutting 
across the traditional divisions of the disciplines can then be seen to be increased if removal of 
the barriers enables science instruction to operate on the relevant levels mentioned above. Again, 
the function of the analysis scheme has to be seen as assessing the value of a structuring principle 
in view of stated criteria (e.g. in terms of relating its quality to a broader field of experience 
than provided by science alone). 

From the method aspect, two main questions arise: (a) is and can a learning method be an 
integrating principle? and (b) is there a method best suited to integration? 

The method aspect shifts the problem from the object side (nature, science, technology) to the 
subject side (student, learning theory, cognition). It might even want to account for the fact that 
an interdisciplinary inquiry might be more the acceptance of the natural human process of 
exploration than a method, but nevertheless lift it to a method status for the operative purpose of 
construction of an integrated science curriculum. : : 

This aspect is related to discussions in the philosophy of science concerning the logic of 
discovery versus the psychology of research, and linked with Dewey's theory of inquiry. The 
underlying assumption of both questions concerns conceptual understanding and is directed ana 
a method such as problem-solving or inquiry as a basis for integration. In this Herat “i 
structuring principle of the subject matter is governed by the learning structure of E E =. 
The analysis scheme should address: (a) the feasibility of methods in view of an ini Kare - 
programme; (b) how to evaluate these methods with respect to their suitability to a C oral 
takes into account the non-reduced scope of science - Science as a process between man E 
nature by means of work; and (c) the integration of science disciplines with associated an 
inherent information such as the political, moral and economic implications. p ái 

When the content no longer distinguishes between "neutral" science and ivalue-oriented 
non-science, when the point-of-view that "science is concerned with value-free statements has 
made room for a more realistic approach, the method will have to account for these approaches. 
An example of the linkage and compatibility of method with content is given in the Humanities 
Curriculum Project of Schools Council/Nuffield Foundation. 

The notion, for instance, of the "neutral chairmanship" in the humanities project (a rather 
basic teaching strategy being one of discussion rather than instruction) provides a basis on which 
the question "what is it about the method that is integrative?" attains a concrete status. Careful 
analysis of the content-method inter-relationship might lead to the statement that it is impossible 
to look at the aspect of integrating content without employing a method that is also integrative. 

This has implications for teacher education, as the teacher's role introduces integrative. 
features by means of chairing discussion groups which adopt the aim of the project. This issue 
of teacher training is naturally relevant also to the content side. Robinson has criticized the 
abstract use of "processes" or "methods" in science education and also stressed the importance 
preparing teachers to overcome the distorted artificial structure in dichotomized science instruction 


by including philosophical considerations of science in teacher training, thus enabling teachers we 
avoid this dichotomy. 1 


of 


1 l 
J.T. Robinson, Philosophy of Science: Implications for Teacher Education. 


Journal of Research: 
in Science Teaching, Vol. 6, 1969, p. 99. 
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Finally, we should be compelled to analyse the role of evaluation in the development of the 
trends in integrated science curriculum construction. House has suggested that "there is a 
natural antipathy between change and evaluation" and that "evaluating something means being 
skeptical ...".1 A penetrating consideration of these two statements could lead one to postulate 
that evaluation of integrated science curricula might negate the further development of endeavours 
in this area. This latter contention can, of course, be refuted if the results of the evaluation, 
although based in traditionality and skepticism, are positive. 

The evaluator, and most importantly, the relationship with the curriculum developer (the 
client) come likewise into play in a consideration of the relevance of evaluation. Tawney 
(Chapter 2) discusses the difficulties of formal evaluation in relation to other groups who can be 
seen as clients of the evaluation. Contrary to the above possible interpretation, House contends 


that; 


"to induce change one must break through the image of the individual, the paradigms of 


the discipline, and the ideology of the organization". 


or is one who dissents from doctrine, who seeks to introduce new 
curriculum developer to inconsistencies or contradictions in a 
thinking about the evaluation of integrated science curriculum 

1 to reflect upon securing evaluators who have a kindred but 

to development in this field. 


A person acting as an evaluat 
information and to expose the 
programme. Perhaps in our 
materials, we need most of al 
nevertheless questioning stance 


of Evaluation. In: R.M. Rippey (ed), Studies in Transactional 
E.R. House, The w-— 0n Publishing Corporation, 1973, p. 256. 
Evaluation. Berkeley: 


2. 
ibid., p. 265. 
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nature and philosophy of science. 


Rutherford, J. and Gardner, M., Integrated Science Teaching. In: P.E. Richmond (ed), 
New Trends in Integrated Science Teaching, Paris: 1971, Vol. 1. 


The authors present a rationale for integration and then consider how content can be : 
Selected for integrated science programmes, what methods are useful in the organization 


of such courses and seven practical considerations which must be taken into account 
when developing such courses. 


Scriven, M., Value Claims in the Social Sciences. 


Boulder, Colorado: Social Science Education 
Consortium, Publication No. 123, 1966. 


Although attending primarily to social science education, this article has much relevance 
to the aspects of science instruction related to values and value judgements. 
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Showalter, V., The Unity of Science. In: What is Unified Science Education? Columbus, 
Ohio: Centre for Unified Science Education, 1974. 


This is a collection of articles describing various aspects of unified science education 
including a discussion of general consideration, terminology, the unity of science and 
guidelines for developing unified science units. 


Smith, M.B., Editorial, J. abnorm. soc. Psychol., No. 63, 1961. 


This short editorial, directed to research in psychology, points out nevertheless, the 
deficiencies in the guidance of scientific development by theory in light of the various 


historical perspectives. 


Stake, R.E., Objectives, Priorities, and other Judgement Data. Review of Educational Research 
(Boulder, Colorado), Vol. 40, No. 2, 1970. 


ins that judgements are an integral part of evaluation. He reviews 


The author mainta 
methods of gathering and reporting such data. 


several sorts of judgement data and 


SSEC: Curriculum Materials Analysis System. Boulder, Colorado: Social Science 
Education Consortium, Publication No. 144, 1971. 


This publication presents the Intermediate form of the SSED analysis scheme. A more 
comprehensive form (The Long Form) for professional evaluators and a shorter form 


(The Short Form) for teacher use are available from the same source. 


Humanities Project. London: Heinemann Educational Books, Ltd., 1970. 

This social science project for students in the age group 14-16, emphasizing the neutral 
chairmanship of the teacher in class discussion, aims at developing an understanding of 
social situations and the controversial issues that arise from such situations. The 
various units (8) and the forms of the materials should be of interest to those developing 


an integrated science curriculum. 


IPN Curriculum Physik, Steureung und Automation (Regulation and Automation). 
Stuttgart; Ernst Klett Verlag, in press. 

This unit is a good example of how non-science content can be integrated with science 
content. It is available only in Germany. 

Man and the Environment. Boston: Houghton Mifflin Company, 1971. 

Man and the Environment 

-vear course using 4 content scheme as the integrating factory. 


A a one 
"dues, de the environment (biological, physical, social, industrial) 


Many of the influences On 
are incorporated. 


The Open University. Bletchley, U.K.: 


The Science Foundation Course Units (34) developed for the Open University courses, 
involving studies in the various sciences, in the history of science and in the relation 
of eh to society, illustrate how a unit approach might be used to create an 


integrated science curriculum. 


The Open University Press, 1970. 
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The reader interested in further information concerning curriculum evaluation might well 
refer to the following: 


AERA Monograph Series on Curriculum Evaluation: Vol. 1: Perspectives of Curriculum 
Evaluation, Vol. 2: Evaluation Activities of Curriculum Projects, Vol. 3: [Instructional 
Objectives. Washington D.C.: AERA, 1967-9. 


House, E.R. (ed): School Evaluation, The Politics and Process. Berkeley: McCutchan 


Publishing Corporation, 1973. 
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Studies in 
Educational Evaluation, Vol. 2, No. 2, 1976. 
Popham, W.J. (ed): Evaluation in Education, Current Applications. Berkeley: 


McCutchan Publishing Corporation, 1974. 


Schools Council Research Studies: Evaluation in Curriculum Development: Twelve Case 
Studies. London: MacMillan Education Ltd., 1973. 


Walberg, H.J. (ed): Evaluation Educational Performance. Berkeley: McCutchan 
Publishing Corporation, 1974. 


A more theoretical discussion of the possibilities and implications of integration of the 
sciences with non-science content can be found in the following: 


Frey, K. in collaboration with Bayrhuber, H., Bloch, J.R., Dumpert, K., Haft, J., 
Hameyer, U., Haussler, P., Jaeckel, K., Kunzli, R.: The Conce t Inte rated Science 
Curriculum. Background paper to the Exeter Symposium on Integrated Science: and 


in; Studies in Science Education, 1976 (in print). 
Frey, K.: Integrated Science Education in the Federal Republic of Germany. Presented 
at; 


Symposium on "An Interdisciplinary Approach Toward the Teaching of Science 


Subjects at the Secondary Level". Exeter, 8-14 September 1974. Strasbourg: Council 
of Europe, 1974 (mimeographed manuscript). 


Fuller, W. (ed): 


The Social Impact of Modern Biology. London: Routhedge and Kegan 
Paul Ltd., 1971. 


Habermas, J.; Theory and Practice. London: Heinemann Educational Books Ltd., 1974. 
Habermas, J.: Knowledge and Human Interests. London: Heinemann Educational Books 
Ltd., 1972. 
Lakatos, I. and Musgrave, A. (eds): 


Criticism and the Growth of Knowledge. London: 
Cambridge University Press, 1970. 


Marcuse, H.: One-Dimensional Man. Boston: Beacon Press, 1964. 


Margenau, H. (ed): Integrative Principles of Modern Thought. New York: Gordon and 
Breach, 1972. 


Mischel, T. (ed): Cognitive Development and Epistemology. New York: Academic Press, 1974: 


Phenix, P.H.: Realms of Meaning. New York: McGraw-Hill Book Company, 1964. 
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5 What skills are needed to teach 
integrated science, and how can 
their development be monitored? 


Clive Sutton 
University of Leicester, United Kingdom 


SUMMARY 

e decision-makers who are in a position to assist the growth of a 
and who therefore need information about current levels of skill. 
(i) teachers individually and in groups, who are consciously 

) inspectors, advisers, and organizers of in-service study; 


This paper is mainly for thos 
more skilful teaching community 
These decision-makers include: 
trying to improve their own work; (ii 
and (iii) those reponsible for initial training. 

It assumes that teacher skills are at least as important as curriculum materials and 
discusses ways of identifying those skills that are 


of integrated science. 2 i 
Trends in the demands made upon integrated science teachers are surveyed and suggestions 


are made for improving the flow of feedback to each of the above groups of people. 


Why skills are of increasing interest. 

hing skills is partly the result of dissatisfaction with the 

culum packages of the sixties. | Members of curriculum project 
d despair when their materials are used, but not in the way 
tion away from the books and apparatus, towards the patterns 
How far, for example, does the teacher encourage the 

S i tic enquiry that the course designers intended? 

—À — ied psa to the question of skills. Firstly, Supposing I am a teacher 
involved in a new programme of integrated science, am I myself skilled in these methods of 
enquiry? How good am I, for example, at suggesting hypotheses, designing experiments to 
test them, interpreting unfamiliar data, in an integrated. science context? Secondly, do I possess 
the pedagogic skills to organize a learning environment in which children will conduct enquiries? 

Some research suggests that even when teachers intend to teach in au eniumng style, they 
may unintentionally exti nguish enquiry by the pupils instead of encouraging it. The pedagogic 
skills seem to be of considerable subtlety. Will I and my colleagues acquire inem without help, 
Or are specific courses required? How much money and effort should be put into developing 
materials on the one hand and developing skills on the other? Certainly because teachers are 


The trend to discussion of teac 
Supposedly 'teacher-proof' curri 
teams have sometimes expresse 
intended, and this has turned atten 
of interaction in the classroom. 


1 
See fo p. Barnes et : 
(Tape cam , and group discussion as a Way of developing understanding of teachers.), and 


M.R. Ro "Wait-time and rewards as instructional variables, their influence on language, 
logic tes control Part one: Wait-time". Journal of Research on Science Teaching, 
Vol. 11 = 2, p 81-94 (1974), also reported in Rowe, M.B., Teaching Science as Continuous 


Enquiry, McGraw-Hill (1973) 


al., Language, the Learner and the School, Penguin, 1969, p. 25. 


particularly required for an innovative programme 
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frequently viewed as managers, or facilitators of learning, these skills are an important focus 
of attention for evaluators. How and when are they acquired, and what factors assist or hinder 
teachers in the adoption of new roles? 


What do we mean by skills? 


The term "skill" implies something that can be acquired or improved. This contrasts with 
personality characteristics, like "intelligence" or "warmth", which so many researchers have 
tried (almost always unsuccessfully) to correlate with pupil achievement. We may extend the 
examples given in the previous section to include, say, "the ability to organize a room in which 
children will learn independently", or "listening skills", or "questioning skills", or "the ability to 
lead a discussion". These are by no means unambiguous in meaning, but they do suggest the 


possibility of practice and improvement. As we shall see below, some workers have preferred 
stricter operational definitions. 


Indian boys working independently with electric circuits 


Diut p cy P Purse of teaching is an analytic process intended to clarify one's thinking 
actually seed: v ves. It could be helpful both in diagnosing how to improve and in 
depending on ina work. The analogy is with a musical performance - complex, variable, 
another. but oa aes iim use of many different sensitivities and skills that interact one with 
cud A: € p of being improved by the practice of quite specific aural discriminations or 
xterities. To describe and practise these does not exclude the possibility that some 
— may achieve the same result by quite unorthodox techniques. 
("the looien Jae ss a common form of questioning in American science lessons 
The structuring of the "e Ri by the teacher are of the order of one second in duration. 
ibis eaa ^. gem is substantially done by the teacher, and pupils respond, at the 
ordo g. in brief phrases, without any complexity of explanation. When teachers 
engthen the pauses to about three seconds, particularly those which occur after a pupil has spoken, 


several chan : 
comments aq pam The Pupils not only elaborate their first answers, but also make more 
Sk more questions on their own initiative. There are more speculative 


"gs i " i : 
uggestions" and more comparisons of one child's ideas with another's, and so on. 


ee If we: assume that such things are desirable at least for some of the time, the correlation 
is interesting , even though we might not be sure that it is the longer pauses which actually cause 
the greater pupil participation. It is a short step to saying that we have here an identifiable 
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skill which teachers could usefully practise. But at what level of generality should we define 
that skill? Shall we call it 'wait-time management! or ‘ability to engage children in group 
discussion! ? The former is more specific, and more amenable to simple practice and training; 
but on the other hand, it is possible to imagine teachers slavishly managing wait-time and getting 
no results, because they cannot simultaneously manage other factors such as the non-verbal means 
of engaging children's interest and building their confidence, the distribution of rewards in the 
group, or the provision of adequate stimulus. These are probably all subsumed in the ‘ability to 
engage children in group discussion’. Perhaps we should tolerate the comparative vagueness of 
this latter term (e.g. just what quality of discussion is implied?) for the sake of the gain in 
reality. " 
If we take the analytic dissection too far, we get so many components that it would never be 
feasible to train anyone in all of them separately. We also risk asking teachers to practise 
things which they cannot afterwards put together into an effective teaching style. On the other 
hand, it could be that many components take care of themselves and that alteration of a few key 
ones could produce large changes in the total pattern of interaction. Rowe evidently thinks this 
is true for managing wait-time and for controlling the verbal rewards accorded to pupils. She 
has shown that teachers can change their behaviour in these areas by practice, with some of the 
results already mentioned. These changes might be establishment of different habits. 
Alternatively, they could be looked at as the external results of ‘inner! changes in the teacher 
- changes in attitude, emotional changes (less fear of embarrassing silences), changes in how 
one sees the job. The way in which we look at them matters a great deal when it comes to 


devising programmes to improve teaching and learning, as I shall discuss in a later section. 


Teacher and pupils work things out together 


1 i i lving many of these skills in varying 

A lls, and of 'teaching' as invo. 

co TE hatto: Of separate A pousemustses for assessment and evaluation of teaching. For 
mbinattons, has consi? n of performance as a profile (rather than a global "good" 


1 descriptio 
im IL Ap makes | arb vds towards self-assessment of one's own work. It is possible 
bad"). It facilitates formance as judgemental of themselves; but by discussing 


f per 
uat. Roschaea fanr sepe pour the non-judgemental stance of criterion-referenced testing 
Particular skills, we can h it is a quality or skill which is assessed, not the person's 


E i ic 
EAE oe — Ta d as a teacher, I may have mastered the skill of translating a 
-all worth. o: 
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recall question into a comprehension question, but may never have attempted to lead a discussion 
on the social implications of scientific discoveries. As a decision-maker in relation to my own 
professional development, I will of course have to "take one consideration with another". If 1 
lack a particular skill, I may take steps to acquire it (or in some circumstances I may defer 
doing so) and build upon others that I possess already. 

Such evaluation is intended to be supportive of teachers in their own professional development 
rather than a demand for conformity to some predetermined pattern of teaching. There is no 
single definitive set of ideal teacher behaviours and it seems unlikely that there ever will be, 
given the great diversity of teachers, pupils, work situations and outcomes sought from a teacher's 
work. At best, the multi-skill view of teaching enables individual teachers to widen their 
repertoire of ways of working. The choice of which skills to use in any particular situation will 


depend on teacher judgement. 


Trends in the demands made upon integrated science teachers. 


Anyone reading the teachers' guides to recent integrated science courses is likely to be struck by 
the very great professional demands that are made upon the teachers. 


Consider the following, from A Guide to the Australian Science Education Project (ASEP): 


"Teachers! views of their detailed responsibilities differ greatly as each interprets his 


or her broad responsibilities in ways consistent with his or her own strengths and the 


particular situation concerned ... As a basic to work from during the early development 
of ASEP, responsibilities of a teacher with respect to learning were categorized into 
four broad areas ..." 


"1. Providing facilities for learning. 
2. Preparing the students for learning. 
3.  Effecting learning, and 


4. evaluating the effectiveness of the learning that takes place and of the learning 
processes ..." 


"The following is a summary of what (trials) teachers regard as being important when 
ASEP materials are being introduced and used ..." 


There follow fifteen pages of comment on such things as knowledge of the broad aims of 

ASEP and their own curriculum goals, thorough preparation of the science room, organization 
of equipment, books and visual aids in the right place at the right time, handling individual and 
yp A yonk with different children doing different things at the same time, knowing when 
exposition is required in the form of formal teaching or demonstration, ability to keep track of 
pupils' progress, providing additional stimulus for the very bright and help and encouragement 
for the veny slow, providing an atmosphere of trust, willingness to modify and improvise when 
ei ul ( bed a teacher's kit of odds and ends in a bag or large pockets"), willingness to 
them fo hial ear cue The ASEP teacher is expected to establish rapport with pupils, help 
Gistmeaat e veg, move about the class and become involved in their activities, manage 

nv ety, communicate, motivate, encourage and assess, guide pupils in the choice of 
activity, be enthusiastic, be prepared to listen, understand individual differences, and plan ahead. 
drum pei is made of the skills of organizing discussions - in the arrangement of furniture, 
guiding pupils! preparation, monitoring the group dynamics of the event, and 'with tact, sensitivity 
and skill', redirecting the discussion if necessary, but without dominating it. An ASEP teacher 
will also have to be skilled in diagnosis beforehand and assessment of what is learned. 


1 3 : j à 
Australian Science Education Project, A Guide to ASEP, Victorian Government Printer, Melbourne. 


56 


What skills are needed to teach integrated science, and how can their develop- 
ment be monitored? 


Elsewhere in A Guide to ASEP, it is clear that a teacher will need to know a good deal about 
children's intellectual development and about Piagetian theory and technique, and to have a critical 
awareness of the strengths and limitations of different types of enquiry. In addition to this, the 
teacher will need to be willing and able to find out about the major biological, chemical, physical, 
geological, sociological and psychological concepts that could be unfamiliar, and able to help 
pupils design scientific methods of investigation and analysis for problems that raise during their 
enquiries. 

The ASEP materials are intended to be supportive to teachers; the aims of serving general 
education and pupils' personal development are ones which are widely acclaimed. The 
methodology of pupil activity, guided, but partly under their own control, while challenging to the 
confident teacher, may be daunting to others. ASEP is possibly more demanding than some other 
integrated science courses, but it is salutary to compare these expectations with those that have 
been made of some single-subject science teachers in the past. Roughly, they were to teach the 
content of the subject as they had already learned it, mainly by exposition, and train the pupils 
in the ability to answer examination questions about Hs ; 

The demands made by the Science 5/13 Project} upon primary and some secondary school 
teachers in the United Kingdom are similar or greater. Many of the primary teachers concerned 
are known to have very little background of science, and the booklets provide a lot of support, 
both in suggesting pupil activities and in providing background information for the teacher. 
Nevertheless, the flexibility implicit in the approach requires that a teacher should be able to 
notice what captures the interests of particular pupils, to discern the teaching potential in these 
interests, guide the children to concentrate on the more investigable questions, and manage their 
enquiries so that useful learning Occurs in one or more of eight general categories of objectives. 
Again this is to be done for groups of children working on different topics in the same room. 

The demands of some other integrated science programmes seem relatively modest, as at 
least the topics to be taught have been chosen by someone else. Her the Scottish Integrated 
Science Scheme, the working party recognized that teachers Were qualified frequently in only one 
Or two separate sciences, and that they might nevertheless be required to teach subject matter 
for which they have no great interest or aptitude" . We recommend throughout this report ... 
the teaching for attributes other than knowledge, and we know Hat we are asking many meg 
to undertake work for which they may have little specific cape Corresponding prot ems 
have been met in the West Indies —. as Project and Malaysian Integrated Science, 

1 i sc H 

both of which ars adaptations ee Process Approach", the American Association for the 
hing E oa t necessary to devise a full "Commentary for Teacnovan and a . 
"Guide for In-Service Instruction" in order that teschera might become sided in "Basic processès 
Such as observing, classifying, measuring, predicting and inferring, and Me Ime Perna 
Such as controlling variables , formulating hypotheses 2 and begun. rr aiu, 
that have these skills by virtue Ld : : 

teachers would ha 2 in the United States, the organizers have 


j i Curriculum Study 
T Ei the piper lead tole preparation modules, several of which provide training 
ised a series of indi 


A how to use questions well, and how to 
à i i rganize a classroom, : 
in pedagogic skills, such as how to o 


Monitor pupil progress. f teachers more substantial understanding of educational theory is 

The trend to demand o Council Integrated Science Project (SCISP), with its 'pattern-seeking 
ZAIN iusizated in ihe perc pore the teacher who uses SCISP will also discuss controversial 
approach'. Like the ASE 


E 5/13, With Objectives in Mind, Macdonald Educational (1973). 
» With Ove 


m Study Individualised Teacher Preparation. Several volumes 


Intermediate Science Curriculu 1 Learning Corporation, Morristown, New Jersey (1972). 


Published by Silver Burdett Genera 
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issues in the classroom and take up a role which was formerly unfamiliar to science teachers 

- that of dealing with questions for which there is not one generally accepted answer. For some 
teachers, discussions of science and its social implications is more than they thought they were 
employed for. 

A comprehensive set of materials for the development of science teacher skills is that 
prepared by the Science Teacher Education Project in the United Kingdom. This bank of 
resource materials and ideas is not linked to any particular science course, either single subject 
or integrated. The project did not publish an exhaustive list of skills needed to teach science. 
However, the suggested activities are all designed to support the autonomy of the individual 
teacher, to encourage flexibility and develop powers of judgement and decision: e.g. "a willingness 
to look for the teaching potential offered by pupils' questions" (STEP Unit ADA/7). 


Table 6. Summary of kinds of demand being made upon teachers of integrated science. 


T. KNOWLEDGE from several sciences, and from educational theory*. 


2. SKILLS OF ENQUIRY - both in information retrieval (finding out what they do not already 
know from books, etc.) and a range of 'science process' skills. 


3. PEDAGOGIC SKILLS. Some of these are interpersonal - ways of relating with onitareng 
some of them are in management (e.g. arrangement of the learning environment, keeping 
records of pupils' progress, devising tests for this purpose); and some are curriculum 
planning skills (e.g. ability to see the teaching potential in the neighbourhood of the 
school, selecting feasible objectives and learning experiences). 


Knowledge and skills are not totally separate. Science concepts may be necessary on 
exercising science process skills. Educational theory may be required, to provide ah 
conceptual apparatus that is indispensible to the development of pedagogic skills. 


Some writers (e.g. Showalter) describe values and attitudes desirable for teachers of integrated 
science. An attitude is a tendency to perceive things in certain ways and to act in certain 
ways, so attitudes may follow from the possession of knowledge. Newly-acquired concepts 
effectively provide a new "set of spectacles" through which the world is viewed and this is 

the basis of the inter-relationship between cognitive and affective aspects of learning. 


‘The introduction of integrated science courses also appears to have implications for the roles 
of teachers. A "single subject" teacher could find a personal identity partly by thinking of 
herself /himself as a 'chemist', or a 'physicist', an expert in the mysteries of a particular subject. 
An "integrated Science" teacher appears by contrast to be more of a teacher and less of a subject 
Deren teaching not in order to transmit the subject but in order to develop the pupils. The 
ipee is justified for its contribution to general education and not for itself alone. The 

integrated science" teacher is expected to bring out connexions not only between different 
branches of science but also between school science and everyday life. Such a teacher is a guide 
in éhdiues into the local environment and an interpreter of everyday phenomena. Bernstein” has 
described the pressures that this places on both teacher and pupil. The teacher is required to 
reveal more of herself/himself and to expect the pupil to do so as well, as cross-references are 


Science Teacher Education Project, Activities and Experiences, The Art of the Science Teacher, 
and Theory into Practice, McGraw-Hill, U.K. (1974) 


B. Bernstein, "On the classification and framing of educational knowledge", in Class Codes and 


Control (1973). (Related to a discussion of constraints on role identification for integrated science 
teachers.) 
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made continually between the lesson content and one's personal experience. Bernstein argues 
that the psychological safety accorded by the old curriculum, in which subjects are isolated from 
each other and from real life, is part of the reason for the survival of such a curriculum. The 
integrated curriculum, he argues, gives less privacy to teacher and pupil, and requires a more 
explicit consensus of values. 


Attempts to develop and monitor skills. 


Rather less work has been done on the development of science process skills than on the 
pedagogic skills. In some countries, there has been a growth of personal project work in 
initial training courses. In the United Kingdom in the sixth form and at college, students may 
nowadays be put in a position where they have to select for themselves a general area of enquiry, 
define an investigatable problem within it, state hypotheses with predictions, design experiments 
and carry them out, and interpret the results. Assessment schedules exist which discriminate 
à person's competence in these skills separately (see bibliography). Procedures more — 
Characteristic of one branch of science than another (e.g. design a control experiment in biology) 
no doubt require experience at least within these different branches. 

By contrast to the lack of growth of process skills, there has been a huge growth io 
techniques for developing pedagogic skills. These techniques include (i) interaction analysis, 
and (ii) microteaching, which have built-in means of monitoring the acquisition of skills. Both 
feature frequently in the movement on the United States for (iii) competency-based teacher 
education. 
g tool has been associated with the names of Flanders, 

claim that very extensive training (of the order of thirty 

eph iiu n - er awareness of Be classroom is changed substantially. 
Others favour alerting student teachers to 'things to look for by means of epis record 
Schedules used for much shorter times. Either way, the first target is y: e students po 
of patterns of interaction and of alternative possibilities. epp miy ater posue a op ing 
4 particular style, in which case the observation schedule can e used as an sm u^ inis 
to show them how" they are progressing. Most users of anian anot analysis E ie = 1 
Specifying preferred interaction patterns, but frequently imply that some are "better" than others. 


(i) Interaction analysis as a trainin 


mply a means of practice for beginners whereby 
properly characterized by two other features, 
-recording), and concentration on a 


(i) Microteaching, though regarded by many 2S 2 
they can teach a few children for 2 short — vae 
namely immediate feedback (e.g. by replay © 


Particular skill. 
" i ishing a mental set, drawing a 
, Y rying the stimulus, establishing ^ 

hui Examples of such skills mae "c ph of examples and encouraging pupil participation. 
In = to a close, ee iio work, one might add items such as ‘selecting examples from 
nm te Dl microteaching session may be preceded by a seminar and by guided 
i Sa ields l ular skill to be practised. then the student teacher RE a " " 
sn & about the particular five) for à short time (say, ten minutes). n this, he or she 

ini-lesson with a few pupils (SaY; A video-recording may then be played 


" chosen skill. 
Makes a deliberate effort to ensibus The teacher may go on to a further session of 
8 x 


i .g. Brown). The costs of hardware 
i :st illustrating the method (e.g i 
mini-teaching, Many texts now vea sche to the rising costs of tutorial visits to schools, 
ae coming down in Western an the technique. However, the substantial consumption of time 
wend — may assist the Lee pe providing groups of pupils readily available in the college tend 
e organizational proble 


E i21 nly a very few skills. i ; 
to UN Student teachers to Led “o microteaching. One is a strict behaviourist approach, 
ere are at least two app 


à be eliminated and new ones 
" ^ habits and that old ones can 
Implying that the 'skills! are pulse ed by appropriate regard from the tutor. As 


_— by this sort of m "or one or two skills per student teacher. The alternative 
loned, this could only be 
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view is that, as with interaction analysis, the method "opens the eyes" of participants to 
possibilities they had not thought of and prepares them to observe other teachers and themselves 
more closely and with more perception. 


(iii) Competency-based teacher education (CBTE) focusses on what it is hoped the teacher will be 
able to do (not just what it is hoped he will know). In 1968, the United States Office of Education 
invited specifications for programmes of teacher education of this type, and nine institutions 
subsequently had their proposals funded. ^ Factors which contributed to the development of CBTE 
programmes probably included the emphasis on behavioural objectives and systems analysis in 
contemporary literature, disenchantment with existing programmes, and particularly educational 
theory courses, discussion of cost effectiveness and pressures for teacher accountability. 
"Competencies" (skills, knowledge, behaviours and sometimes attitudes) which it is hoped teachers 
will develop are derived by the CBTE designers from many sources. They may work from a job 
analysis, from "expert opinion? on what teachers will be required to do, from indications in 
empirical studies that particular skills may be linked with particular pupil gains, or else they may 
proceed *logically? (from philosophical assumptions about the nature and purpose of teaching and 
learning) to specify what competencies should be fostered. 

Those concerned with integrated science courses which have a distinctive philosophical base 
(e.g. Science Education Program for Africa) might find this approach useful. However, there 
are a number of cautions to be made. The possession of a list of competencies appears to 
simplify the evaluator's task, since at least the desired outcomes are declared. On the other 
hand, it could draw attention away from unexpected and unintended outcomes, and from the process 
by which student teachers are learning. In my view, it certainly oversimplifies the complexity 
of meaning of "effectiveness" of teaching. . : 

Table 7 ‚over page , presents some arguments for and against CBTE programmes, in such a 
way as to raise questions about what evaluators concerned with new programmes in integrated 
science might try to monitor, in addition to the specified competencies. 
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Table 7. 
for evaluators. 


What skills are needed to teach integrated science, and how can their develop- 


ment be monitored? 


Some arguments for and against CBTE programmes, and corresponding considerations 


a eS 


For 


Against 


Notes for evaluators 


1. By identifying skills 
which an intending teacher 
may practise and master, 
CBTE allows beginners to 
concentrate on one thing 
at a time, instead of 
facing them with the 
immense complexity of 
the whole job. They may 


gain confidence rapidly. 


Impractically long lists of 
microskills may have the opposite 
effect. Also, to maintain 
confidence the beginner will have 
to know that he can use particular 
skills in the more complex context 
of whole teaching and not just in 
isolation. 


Evaluators need ways of 
monitoring skills not only 
separately but in the context 
of whole teaching of the 
innovative programme of 
integrated science. They might 
also attempt to follow teachers 
through initial and in-service 
courses, examining their 
motivation and response to the 
various learning tasks. 


2. The emphasis on student 
performance assists the 
tutor out of the role of 
information-giver and into 
the role of a coach who 
helps the students to 
practise and develop their 
Own competence. 


3. The emphasis also on 
what it is hoped pupils will 
be able to do assists the 
Student teacher out of the 
role of information-giver 
and into the role of helper 
Or manager of children's 
learning. 


4. Clarity of description 
Should help both tutors and 
teachers to demonstrate 
accountability to themselves 
as professionals. 


This emphasis may encourage 

the belief that teacher preparation 
is just the acquisition of a set of 
behaviours, ignoring the likelihood 
that such behaviours are related 
to tinternal' qualities - attitudes, 
concepts, ways of looking at the 
job. It could result in laborious 
and ineffective attempts to 
acquire behaviours which would 
follow easily from changes in 

the student's cognition or affect. 


of the job may diminish the 
teacher as a professional, 
responsible for decisions about 
ends and means, and tend to 
make him or her into a technician 
applying specified procedures to 
produce given results. 


There is some political danger 
in drafting lists of competencies 
which become the focus of power 
struggles between say 
administrators and teachers, 

or parents and teachers. 

Within a CBTE framework of 
thought, it may be somewhat 
easier to dictate what teachers 
or institutions shall do. 


the job, and particularly changes 
in this in response to different 
experiences, are an important 
area of enquiry. When a 
change to an integrated science 
programme is made, changes 

in their view of their own role 
should be examined (see Combs). 


How teachers make decisions 
about their work is possibly as 
important a topic of enquiry as 
what skills they have. 
Attention should be given to 
identifying the area of freedom 
that an integrated science 
teacher (a) has, (b) thinks he 
has. 


innovative programme will 

require a description of the 
social and political context 

in which it is occurring. 


a aes 
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Suggestions for action. 


Certain skills of enquiry and associated concepts used in an integrated science context may not 


have been developed in traditional single subject courses which the teachers experienced in their 
own early education. 


Almost certainly, also, some requisite pedagogic skills will not have been 
acquired during their training. Hence: 
1. Teams of developers, in conjunction with teachers in a particular region, might usefully 


compile a statement of what skills are thought necessary to put the integrated science 
programme into effect. 


Lists of such skills should form a guide to be borne in mind by 
teacher educators and others. 


There is some value in attempting to state these in terms 
of what teachers are expected to be able to do, or at least to illustrate the desired skills 


by giving examples of what teachers might do. However, the possession of particular 


skills should not be mandatory upon teachers, as there is, as yet, no evidence of the 
indispensibility of any one skill. 


Trinidadian teachers involved in curriculum development 


dass d Ml 


Programmes to help teachers develop and improve their skills should not be launched without 
Some consideration of the assum 


ptions underlying the learning methods that are proposed. 
For example: 


(i) It might be thought that relevant skills can be acquired simply by practising them until 
they become established as habits (behaviour modification). 
(ii) 


An alternative view is that the behaviour of a teacher changes only in response to 
changes in the concept of the job and its purposes. 


1 
(A viewpoint elaborated by Combs 2 


1 
Arthur W. Combs, The Professional Education of Teachers; Allyn and Bacon Inc., Boston (1965): 
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What skills are needed to teach integrated science, and how can their develop- 
ment be monitored? 


There are several techniques intended to increase understanding of or to give practice in 
particular skills and which also provide a means of evaluating progress. They include: 
recording one's own lessons for subsequent discussion in groups; discussion of lesson 
transcripts; simulation and role play; interaction analysis; and microteaching. The 
cheapest and simplest of these is probably discussion of lesson transcripts and a brief 
description of such a method for an in-service group of teachers is given by Barnes. 

It was developed with groups of serving teachers, who discussed tape recordings and 
transcripts of lessons. In the process, they became able to attend to aspects of teachers' 
and pupils! language which had previously been unnoticed. Another programme is described 
by Adelman and Elliott!, also working with serving teachers. These teachers were asked 
explicitly to consider whether in their classrooms there was evidence about a number of 
suggested hypotheses linking particular teaching skills with the children's readiness to 
enquire. In both cases, the teachers concerned seem to have acquired a new way of 
thinking about and looking at their work, i.e. a change in understanding came first, and the 
writers seem to think that this prepares the way for the development of new skills. 

The interplay of these two aspects is of particular interest. The increasing attention 
paid to "skills" in recent years had been at the expense of considering the relationship 
between knowledge and skills. There has been a tendency to think that the skills can be 

n a desire to get away from ‘old-fashioned! teaching of 


acquired in themselves, perhaps i i 
duse amounts of information and concepts. Teacher educators are talking more about 


what people can do and are less keen to be seen worrying about what they know. The 
question of whether doing can occur without knowing has meanwhile been avoided. Teachers 
can obtain feedback about their own skills not only by these techniques but also by careful 
monitoring of what their pupils can do, using the methods developed by Harlen. (See also 


Duckworth's checklist in Harlen's paper in this volume.) 
ade to ensure a flow of feedback to all relevant decision-makers, in 


A m 
pego eal emental of individual teachers. 


a form that is not judg 
be given by curriculum planners to the constraints limiting the 

j i i the teacher's self-concept and 

t skills. These might include p 

S Tars authority pattern of the culture and pupil expectations of teacher 
b reci pre ts to help teachers develop skills should take account to these constraints. 
i om ^. ree of integrated science teaching in a community where separate 
ne raten e e already well established, and where the teachers have themselves as 
E ject teaching erienced an integrated study, may be extremely difficult. It might 
e NM of some integrated science courses in the universities and colleges, 
equire e intro 


and the preferential selection of teachers from these. 


Consideration should also 


i n Teacher Education for Curriculum Reform", An interim report on 
-3 eigen ad » re Me Project, British Journal of Teacher Education, Vol. 1, No. 1, 
work oj e For 


January 1975. 
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6 Evaluation instruments for integrated 
science teaching 


Victor J. Mayer and James M. Richmond 
Faculty of Science and Mathematics Education, Columbus, United States 


SUMMARY 

-collecting procedures useful in evaluation studies of integrated 
Science teaching. It focuses especially upon objective respondent-completed instruments 
identified in a world-wide survey of science educators. The instruments are categorized and 


representative examples of each category are discussed. Recommendations are suggested for 


instrument use and development. 


This chapter briefly reviews data 


Various data-collecting procedures have been developed for use in evaluation studies that are 
adaptable for use in integrated science. The decision as to which procedures are to be used in 
a given study will depend upon several factors, including the nature and use of the information 
desired, the size of the implementation Or development project, the approach to be used in 
evaluation and the resources available to the evaluator. Procedures that have been extensively 


used include interviews, classroom observation, free-response forms, projective techniques, 
questi i lists. 
ani, RM eee have the advantage of providing a wide variety of detailed 
information. Interviews, for example, allow the evaluator to probe teachers for potential 
Problems implicit in the curriculum materials being used and to follow-up unanticipated 
Charüctarieticn of thë implementation effort as raised by the teacher in the interview. Such 
n formative stages of evaluation, when the developers 


i ost useful i 

igi eap n viden the evaluation and to be using the resulting information 

directly in the revision of materials. Such intensive data-collecting techniques , however, require 

much staff time and would be extremely expensive to utilize on a large scale, especially in the 

Summative phases of evaluation. They also yield information that may be suspect, because of 

Possible bias on the part of the data collector or as a result of egocentrism on the part of 

respondents to interviews and Open response forms. pt addition, the presence of an outside 

element (the data collector) in the classroom, or the interaction of strangers with the teacher 

9r child may alter the context to such S3 extent E vis eee obtained is not representative 
" iculum 18 i 

of "e Saal climate in whit, “oramination of data-collecting procedures has been limited to 

respondent-completed paper-and-pencil forms that Are capable ot 7 doling id mingle Bere it dm 

or more separate factors. They are what are referred to in this paper as "instruments" - . 


servation systems" is referred to the comprehensive 
r.p. Evans and P.E- Blosser, A Review of Research on Teacher 
jence Education, Association for the Education of Teachers of Science 
thematics and Environmental Education Clearinghouse, Columbus, Ohio, 


The reader interested in nglassroom ob 


review by A.L. Balzer, 
Behaviour Relating to Se 
and the ERIC Science, Ma 
1973, p. 396-447. 
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The data-gathering instruments discussed herein cannot provide as rich and detailed information 
as the more informal techniques. But they are less expensive to use in terms of staff time, and 
therefore can be used with a larger population to yield a wider base of information. Also, by 


using techniques such as matrix sampling, the amount of information obtainable from each 
administration of such instruments can be increased. 


Further, by using properly developed and 
administered instruments, objective and representative information can be obtained through the 


elimination of observer bias and by the removal of outside influences that might be introduced into 
the classroom by use of informal data-gathering techniques. 

The purpose of this paper is to provide the reader with an understanding of the wide variety 
of evaluative instruments that have been developed and the broad range of information which can be 
obtained through their use. To this end, the types of instruments available have been categorized 
and examples selected for description from each category. 


An additional function is to identify instruments which can be used as models for the 
development of other instruments for use in evaluation studies. 


In a few cases the actual 
instrument itself might be useful. 


i Information is provided in Table 8 and in the list of references 
that will allow the reader to locate existing instruments. 


The instruments discussed were selected from several sources: 


Unpublished Evaluation 
Instruments in Science Education: 


A Handbook! , which describes 293 instruments; responses to 
a worldwide questionnaire survey of science educators conducted in June 1975 which yielded an 
additional thirty-five instruments; and from a review of science education literature since the 
E of the Handbook. Instruments were selected according to the following criteria, 
namely: 


1. Availability, i.e. the instrument is available from one of several sources - ERIC microfiche 


collections, a journal article, directly from the author or his institution or - where the 
Source is a dissertation - through University Microfilms. 


General Usefulness, i.e. many instruments are available that have been designed to evaluate 
local curricula or to determine cognitive or affective outcomes of specific instructional media. 
Unless such instruments presented a novel format or design, they were not selected. 


Usefulness in evaluation integrated science curricula. Most of the instruments selected are 


listed in Table 8, over page , along with selected characteristics, author and references to the 
bibliography. 


l3 
Victor J. Mayer, Unpublished Evaluation Instruments in Science Education: A Handbook, 
Center for Science and Mathematics Education, The Ohio State University, Columbus, Ohio, 1974: 
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Table 8. 


List of selected instruments. 


Evaluation instruments for integrated science teaching 


ee. 
- Reliability Validity 


Title 


Student Characteristics 


Mathematics Skills Test 
Scientific Reasoning Test 


Critical Thinking Battery 
Home Attitude Instrument 


Student Performance 


Science Process Inventory 


Instrument 
Science Knowledge 


Concepts 
Concept-Process Test 
Fyffe Process Test 


Test of Science Processes 


Test on Scientific Attitudes 


Inventory of Societal Issues 
A Test of Interests 
Attitudes to Science 


Climate or Process 


Earth Science Test 
Test of Knowledge of SAPA 
Science Support Scale 


Items 
Understanding in Science Test See text See text Concurrent 
Science Reading Test * Multiple-choice 0.90-0.92 Content 
Multiple-choice 0.96 Content Concurrent 
tent 
Scientific Curiosity Inventory Statement sets 0.85 Conten 
Multiple-choice 0.67 Content 
Multiple-choice 0.90 Content Concurrent 
Likert-type a - 
s Content 
Science Experience Inventory Statements 0.93 
T Content 
Test of the Understanding of Science Multiple-choice 0.76 
Statements 0.86 Content 
i - 0.71 Content 
Test of Social Aspects of Science Likert-type pie 
t i - 0.76 onten! 
Abridged Scientific Literacy Likert-type 
0.73 Content 
Pictorial-Aural Inventory of See text 
iple- 0.90 Content 
Level of Understanding of Science Multiple-choice 
Multiple-choice 0.83 Content 
Multiple-choice - Concurrent 
Multiple-choice 0.91 Content 
^ : 0.35-0.71 Content 
Basic Science Processes Test Multiple-choice vagi 
Multiple-choice 0.55 on: 
s 0.70-0.77 Content 
Cognitive Preference Examination Il Multiple-choice e E 
Beliefs About and Attitudes Toward Multiple-choice s 
Si 
cience and Scientists Scale Mtm ——— 
Likert-type 0.75 Construct 
Likert-type 0.69-0.91 Content 
0.77 Content 
Science Classroom Activity Checklist Statements s vires 
ts è 
Science Classroom Inventory Statemen! emis 
Multiple-choice 0.88 onteni 
Multiple-choice 0.68 Content 
Likert-type 0.84 Content 
Statements 0.93 Content 


Rating Scale of Objectives of Science 


Author 


Tisher and Dale 
Heath 

Denny 

Campbell 

Lucas 

Cillizza 
Starring 

Ulhorn 


Cooley and Klopfer 
Welch 
Korth 


Richardson and 
Showalter 


Finkelstein 


Helgeson 


Disinger 

Fyffe 

Tannenbaum 
Beard 

Nay and Kozlow 
Atwood and Wright 
Champlin 


Steiner 
Meyer 


Brown 


Sagness 

Shay 

Mayer, et al 
Merkle 
Schwirian 


Berty 


Education 
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DESCRIPTIONS OF INSTRUMENTS 


In this section, examples of instruments will be discussed in three categories based upon the 
broad types of variables they purport to measure: student background characteristics; student 
performance variables; and climate or process variables. 

The first group of instruments consists of those that attempt to assess student background 
characteristics that are probably stable for the purposes of a particular investigation; e.g. stage 
of intellectual development. They form the student context in which a curriculum is being 
implemented or tested. | Knowledge of such variables is important in both formative and summative 
evaluation stages in order to specify which student characteristics influence the degree of success 
or failure of the curriculum implementation effort and to provide the developer with guidance in 
constructing materials and instructional strategies appropriate for different kinds of students. 

The second group of instruments measures the output variables of a curriculum; the types 
of student performance the curriculum is designed to elicit, such as science knowledge and 


attitudes toward science. The instruments reviewed in this category may be useful in summative 
evaluation stages, where there may be an attempt to differentiate between the effects of two or 
more curricula. In the formative evaluation stages, the instruments may be useful as models 


for instrument development. 
The third group measures process or climate variables: the characteristics of the classroom 


environment in which the curriculum is being implemented. This category of instruments is 
useful in summative evaluation and can also provide models of instruments to be developed for 
use in formative evaluation, to provide control over the degree to which the classroom 
environment supports the objectives of the curriculum being evaluated during its implementation. 


Instruments that assess student background characteristics 


This first group of instruments relates to those student characteristics concerned with a student's 
ability to learn from science instruction and materials; characteristics such as Scientific 
reasoning and critical thinking, which may in turn be affected by science instruction but probably 
not appreciably in the time framework of the typical science course. These nie ont 
although they may fall within the domain of objectives suggested for science instruction, are of a 
broad nature and if affected by instruction, they are affected by the total school context and/or 
are contributed to incrementally by instruction over a long time-frame (years, rather than 
semesters). Therefore, for the purposes of most evaluation efforts in integrated science, they 
can be thought of as relatively invariant characteristics of the student. 


(a) Level of cognitive development 


The Australian Science Education Project (ASEP) materials have been designed for use at 
three levels of student cognitive development. Stage 1 materials relate to Piaget's concrete 
level of development, Stage 2 are suitable for students in the transitional period between concrete 
and formal operations and Stage 3 materials deal with hypothetical situations and are designed 
for students at Piaget's formal operations level. Tisher and Dale have developed a pencil- 
and-paper instrument called Understanding in Science Test! which may prove to be o condi 
the science teacher as a quick and efficient method for determining the level of cognitive 
development of the students in his classroom. ^ The test, which takes about forty minutes to 
administer, provides a measure of the student's understanding of physical science concepts in 
four broad conceptual areas, namely, reflection from a plane surface, the principle of the 
balance, balancing volumes of liquids, and the projection of shadows. Before responding to 
the test items, the students are either provided with a demonstration of the four sets of apparatus 
in question, or shown diagrams of the apparatus by means of an overhead projector. The ] 
Understanding of Science Test was validated with a sample of fifty-seven Australian students in 


1 . 
R.P. Tisher and L.G. Dale, "Understanding in Science Test", Australian Council for 
Educational Research, Frederick Street, Hawthorn, Victoria, 1975. 
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grades 7, 8 and 9, ranging in age from 12.0 to 16.4 years. Since the instrument was developed 
as a pencil-and-paper alternative to the clinical interview, all subjects who took the test were 
also interviewed on "invisible magnetism", "equilibrium in the balance", and "combination of 
chemicals" as described by Inhelder and Piaget. The authors reported a close agreement 
between the two methods of cognitive classification; however, the pencil-and-paper test tended 

to classify the students slightly higher than did the clinical interview. 


(b) Reading and mathematical ability in science 


Several types of instruments have been developed to assess student reading or mathematical 
t 


abilities related to science. | Examples of such instruments are the Science Reading Tes 
(which assesses achievement of selected reading skills as applied to science) and the Mathematics 
student performance at three cognitive levels on ten basic 


Skills Test? (developed to measure É T s 
mathematics skills related to chemistry learning). The mathematical skills are computation, 
Signed numbers, use of parentheses, fractions, decimals, exponents, percents, one-variable 


equations, ratio and proportions, and graphing. Both were developed specifically for use in 
American school situations. 


(c) Intellectual characteristics 


A certain degree of confusion seems to exist in the literature concerning the identification 


and classification of intellectual characteristics of students that affect science learning. The 
following three instruments, however, offer ways of measuring what would seem to be somewhat 


distinct characteristics: 


i 3 
(i) The Scientific Curiosity Inventory 


s ing various aspect: 
tudents express concern ng dent is asked to answer yes or no to each 


st ithi h set the respon! Saba 
: NE Eie two questions which are posed at the beginning of the set. 


assesses the degree of curiosity that junior high school 
s of science. It consists of seven sets of 


sess the respondent's ability to judge the 


"m 4 ts to as 
ii 1 se: i t urports 
(i) The Scientific Reasoning Test P x d to comprehend scientific writing. It 


PRT 7 ign experiments an 
validity of evidence, to design NS respondent is given written descriptions 


A S i ich the 

consi i situations in whic É 

€ wade of experiments or observations. The respondent then answers a 
conce ng Xp. 


ituation. 
Series of multiple-choice items based upon the si 


i illi tions, history, English 

ii ae a B loped by Cillizza, has four sec ` 3 . 

is Critical pepe Tec pes asian of sixty multiple-choice items, subdivided into 
nce, and mathe , 


TUAE i i icular content area. Each 
n itical thinking skills in the particu 
expen ome seas de poet for use with children in English-speaking countries; 
e three instrumen! 


the United States and Australia. 


(d) Home background and past experiences 
to the effects of student home environment upon their 


: $ tors 
The lack of attention paid by evaluate s evidenced by the paucity of instruments developed 


ability to profit from a particular curriculum 1 
lm cit., p- 110. 
ayer, Unpublished Evaluation --- op. cit., 


2. 

ibid., p, gg, 

3. " 

lbid. , p. 107. 

Arthu M S tifi Reasoning Test", School of Education, Flinders University of 
r M. Lucas, "Scientilic 

South Australia, Bedford Park, 1969. 


5 
1 it., p. 104. 
Mayer, Unpublished Evaluation -— op. cit., P 
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in this area. The Home Attitude Instrument! is the only one found which is designed to assess 
the influence of the home environment upon the attitudes of students toward science and scientists. 
The Science Experience Inventory 2 was designed to determine the types of science-related 
experiences that were a part of the student's background. Both were developed for use in 
American schools. 


Instruments that assess student performance 


In this section, instruments are reviewed which purport to measure those aspects of student 
performance which are generally thought to be amenable to change through relatively short-term 
science instruction. These include instruments assessing attitudes toward science and scientists, 
knowledge of science and ability to use science processes. This is where the greatest 
developmental activity has occurred, particularly over the past ten years, when there has been 
increasing cognizance of the function of science curriculum in the development of attitudes toward 
science and in the development of the ability to use problem-solving procedures. The evaluation 
of nationally-supported curriculum development efforts, such as those of ASEP and of the variety 
supported by the National Science Foundation in the United States, seem to have been largely 
responsible for this flurry of activity in instrument development. 


(a) Understanding the nature of science 

A number of instruments have been developed over the past fifteen years which attempt to 
assess a respondent's understanding of various aspects of the scientific enterprise. The n 
that has been most widely used on a worldwide basis is that published by Cooley and Klopfer in 


1961, entitled the Test of the Understanding of Science?. The — E EN 
assessing respectively the understanding of the scientific enterprise, the eie deci - 
scientists and the methods and aims of science. An instrument with similar obje , 


À : k d adults. 
Science Process Inventory *, was developed for use with American High School queen o 

The respondent is asked to agree or disagree with each of 135 statements cii sese of 
Scientists. Korth developed an instrument which focuses specifically on the inter 


Science and society, for use with American students. The Abridged Scientific Literac 


Instrument was developed to assess attitudes toward science and understandings of inter- 
relationships in science. The instrument consists of thirty-four situation-establishing ain 
each with a seven-point Likert-type-response scale. It was used with American high-schoo! 
graduates enrolled as college freshmen. 


(b) Achievement in science knowledge 


A wide variety of instruments, many of which have been published commercially, i = 
developed to assess student levels of knowledge in various science disciplines and in oe 
generally. Those published in the United States are reviewed in a publication by Wall an 


Lipid. p. 99. 
2. 
id. 


3 ins isti jonal 
Janet Wall and Lee Summerlin, Standardized Science Tests: A Descriptive Listing. Nation 


Science Teachers Association, Washington, D.C., 1973. 


4 
Mayer, Unpublished Evaluation ... op. cit., p. 156. 
bide; p- 153. 


6 bia. ; p. 155. 
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Summerlin!. Most of these instruments have evolved out of a context of teaching the various 
courses or as units of a general science course, and therefore 


Science disciplines as year-long 
except perhaps as sources for 


are of little use in evaluation of integrated science programmes, 
items. 


In an attempt to reduce the influence of general reading ability upon performance on a science 


content test, Finkelstein developed the Pictorial-Aural Inventor: of Science Knowledge“ to assess 
the achievement of American Fifty-graders in science knowledge. It consists of sixty 
multiple-choice items using pictures instead of words for both item stems and distractors. 

The newer curricula have increasingly emphasized the learning of science at the higher 
cognitive levels: application and comprehension, for example, rather than just at the knowledge 
level. Helgeson, in studying concept development of American elementary School children, 
developed an instrument? to assess achievement in understanding of certain concepts of force at 
each of the three levels of knowledge, comprehension and application. Jeffrey, in his case study 
Evaluation (see Chapter 12) discusses the use of 


of the Scottish Integrated Science Syllabus 
recall, comprehension, application and 


instruments which assess achievement at four levels; l 
highest ability, which were modified from the Bloom categories. 


(c) The ability to use scientific processes and to engage in problem-solving behaviour 
er the past decade, there has been increased 


red curricula ov 
y scientific process skills and problem-solving 


With the advent of process-cent : : 
activity in attempts to assess student ability to apP. Sees z n 
behaviour. Bs has especially been true at the elementary and junior high or middle school 
level. As a result, a variety of instruments have been developed to assess such abilities. 
Fyffetg instrument 4 dE designed to assess the process-skill acduisttton: of elementary school 
children for the processes of formulating hypotheses and defining operationally, as defined in 
the AAAS Sci E A Process Approach SAPA) materials. Test of Science Processes? was 

cience - zoe ts to use the science processes of 


used to assess the abilities of junior high school studen 


VEGA i i ti inferring and 
Observi i ifvi uantifying, measuring, experimenting, ng 
Tving ,, comparing, sy csipa ien the effects of verbal ability on test achievement, Beard 


pre H " M = 
dicting. Attempting to minim! dio-tapes rather than the written 


€ i i slides and correlated au 
eloped an instrument which used e Processes Test. It too was designed to assess 


multiple-choice type item in her Basic nar the AAAS 
achi h : defined by 7 
Miro ns tb: science procena a t to assess both concept and process development in 
Ohio singer developed a single Process Test" was constructed using as a model 
a. ine high school sape and tems in the processes of science identified in 
ven basic conceptual SC 


USA) Science Teachers Association. 
the 1964 Curriculum Committee Report ©} 


rumen 
His Concept- 
five major i 
f the National ( 


(d) Scientific attitudes 
cience has been the development of scientific 


3 m ation in S 
A major objective of peres looks at information or situations. Nay and Kozlow 
as : 


Attitudes; the way in which 


Wall and Summerlin, op. cit- 


? i 70. 
Mayer, Unpublished Evaluation --- op. cit., P 
3. 
ibid. p, 71. 
4.) 
ibid. , p, 90. 
5. 
ibid. , p, gg, 
op- cit., P- 89. 


6 
Mayer, Unpublished Evaluation —— 
T. 
lbid. , p. 85. 
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developed the Test on Scientific Attitudes! to assess such scientific attitudes as critical-mindedness, 
suspended judgement, respect for evidence, honesty, objectivity, willingness to change opinions, 
open-mindedness and questioning attitude. The instrument was designed for use with senior 

high school students in Canadian schools. 


(e) Cognitive preference 

Preferred learning styles of students have received some attention by curriculum evaluators. 
Atwood2 developed an instrument to assess the degree to which chemistry courses affected a 
student's cognitive preference and mode of attending to information, and whether these effects 
were consistent with the objectives of the course. Tamir, et al, have developed the 
Agriculture Cognitive Preference Test and the Biology Cognitive Preference Test for use in 
evaluating the implementation of science curricula in Israeli schools 3. Despite their other 
relative merits, each of these assesses cognitive preference relative to instruction in a specific 
discipline. | Atwood went on to develop the Cognitive Preference Examination - m4 (CPE-II) 
which utilizes content from general science and social science. Three preferences are assessed: 
memory or retention of factual information, application of information or what the information is 
"good for" and questioning or the challenging or criticism of the information presented. The 
instrument was designed for use with students of the senior high school level. The CPE-II was 
revised, updated and revalidated for use with college science majors by Wright. Any evaluator 
interested in using cognitive preference as a criterion should first consult a comprehensive 
critical review by Brown? in which serious questions are raised as to the nature of cognitive 
preference and the validity of instruments designed to assess it. 


(f) Attitudes relating to science 

The increased concern among science educators for the role of science curricula a 
Students! attitudes toward the science establishment and its workers is reflected in the great 
number of instruments that have been developed over the past ten to fifteen pome: s 

Champlin recognized a problem encountered by others when developing instruments to I 
attitudes. He attempted to construct a scale which would allow the investigator d ae ia 
the influences of knowledge about science and the scientific enterprise from an individual A 
attitude toward them. His instrument has two parts: "part one" assesses the yesgongents 
beliefs about science, and "part two" measures the respondent's evaluation of those beliefs. 
Attitude toward science, then, is determined through a complex scoring procedure relating 
responses on "part one" to responses to items on "part two". The instrument, Beliefs About 


1 
Marshall A. Nay and M. James Kozlow, "Test on Scientific Attitudes". Department of 
Secondary Education, University of Alberta, Edmonton, Alberta, 1973. 


2 à 
Mayer, Unpublished Evaluation ... Op. cit., p. 100. 


5 Pinchas Tamir and Moshe Simanovsky, "Agriculture Cognitive Preference Test", De Shalit 
Science Teaching Center, The Hebrew University, Jerusalem, 1975; and Pinchas Tamir aaa 
Rostam Mukades, "Biology Cognitive Preference Test", De Shalit Science Teaching Center, 
The Hebrew University, Jerusalem, 1974. 


4 É 
Mayer, Unpublished Evaluation ... Op. cit., p. 101; and Robert Wright, "Cognitive Preference 
Styles of College Students Majoring in Science, Mathematics and Engineering", Unpublished 
doctoral dissertation, The Ohio State University, Columbus, 1975. 


5 ies i 
Sally A. Brown, "Cognitive Preferences in Science: Their Nature and Analysis", Studies in 
Science Education, Vol. 2, p. 43-65, 1975. 
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and Attitudes Toward Science and Scientists Scale!, consists of thirty-two multiple-choice items 
in each of the two parts. The scale was used with ninth through twelfth graders from a variety 


of American socio-economic settings. 


The Inventory of Societal Issues? was developed to assess the attitudes of high school seniors 
toward socially significant science-related issues. Using factor analysis, seven interpretable 


factors were found relating to environmental issues and society and the individual's role in these 
issues. : 
f science orientation, attitude to science as a 

A Test of Interests? assesses the factors o xh à S H : 
process d ine to science as a school subject. Additional factors include interest in science 
as a leisure process, as a method of inquiring and as a body of knowledge; and non-scientific 
interest in literatura, fine arts and methods of solving problems by consulting books , teachers 
and specialists The instrument was developed for use with Australian students in the 10 to 


15 year age range. 
Brown developed an instrument 
Scottish Integrated Science Curriculum. 


to assess the achievement of five affective objectives of the 
The instrument has five scales (corresponding to the 

E f the inter-relationships between 
i i i i i d to measure: awareness o! 2 
2e cusricuii Es cwn : ber. awareness of the relationship of science to other areas of 
the UE iade mim A arenei of the contributions of science to the social and economic life 

ties Sursicuiums Et j ent in science; and objectivity in observing and in 
z^ noice ruere a oe used in the development of this instrument? can be used 
a i i approac d 
ac E S e n in evaluation studies. Its ileal and the results 
PART R Chapter 13). 

Obtained with its use are described in Brown's paper (see P ) 


i aviours 
(E) Instruments assessing a variety of student behavi: 


i f 
" iewed for this paper were designed to UE several types [o 
stent beheriomr. An example is ona derloped by the Gebet Com MALE 
Project saitei The Special Project Examination in Inte rate aan Fs sical, information to the 
It attempts to measure student mental skills, ranging ee E It also assesses atiinded 
Performance of complex mental tasks including problem- éd glasas erophaeis igen, the 
Such as willi s, scepticism and concern: T eae isition of knowledge or particular 
achi Mies ue bjectives rather than on the acquisiti f tudent behaviours 
ievement of aims and o jec working model of an evaluation scheme for s en 2 
ipea ARE of integrated science teaching. The 1973 version of the 


i two of which include multiple-choice questions, with many 
examination consists of five sections, tations of data. Each of the other sections contains 
A 


based ther represen : i as available. 
Eod b Mr No item analysis information was 
y qu . 


iables 
um or process varia 
ee EMG s be concerned not only with the results of instruction; data 
to be € 


is of the nature of the implementation effort 
A i rmit an analysis ©! natu r wm 
must also be collected which ia consistent with the objectives es —À e 
and the degree to which this effo dict nih "process" variables. "d : ppe s, failure to 
es little information is pis 1o not e&elly interpretable. Is this failure 
Chieve objectives of the curric 


Evaluation efforts in science need 


i . cit., p. 119. 
Mayer, Evaluation Instruments .-- OP 


ibid 
“+ P- 147. ; 
35 is f Interests" Jaceranda Press, Milton, Queensland, 1969. 
*K. Meyer, "A Test of In. d : 
1 :ectives in an Integrated Science Curriculum", Unpublished 
Sally A. Brown, "Affective Obje Ngtirling, 1975: 


doctoral dissertation, University © 
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curriculum? Is it the result of the failure of teachers to understand the content? Did teachers 
have attitudes favourable to the curriculum? Did they use teaching practices consistent with the 
philosophy and objectives of the programme? A large number of instruments designed to assess 
Such process variables have been developed. Some of the instruments discussed under the 
previous category can also be used for this purpose. 


(a) Classroom activities 


Kochendorfer 1 developed an instrument to assess the degree to which biology teachers were 
using classroom teaching methods which were consistent with those recommended for use in the 
Biological Sciences Curriculum Study (BSCS) programme. ^ Sagness extensively revised 
Kochendorfer's instrument to make it applicable for any science class. ^ His Science Classroom 
Activity Checklist? is designed to assess the degree to which a teacher is using methods and 
techniques that are consistent with the consensus of science educators in terms of effective 
Science teaching methods, methods which facilitate student understanding of the nature of science 
and the scientific enterprise. Two versions of the checklist were prepared. One was designed 
to assess the nature of classroom activities which teachers felt should be used for secondary 
School science instruction. The other determined the nature of activities teachers actually use 
as perceived by their students. The checklist consists of sixty "yes-no response" statements of 
activities. Eight subscales were identified. However, a factor analysis performed D 
slightly-modified Spanish language version of the checklist completed by secondary solenca 
students in Costa Rica, failed to confirm the subtests as factors. In this study, Bon 
identified four factors: the integration of the use of the science laboratory with the science 
course, the use of the textbook and reference materials, teacher-centred teaching and the use 
of problem-solving procedures. i 

Shay, concerned about student-centred instruction in science, developed the Science Classroom. Cieero 
Inventor? to be used in assessing individual preference for the student-centred, non-direct Boienee 
classroom. He developed a criterion model of such a classroom. A pool of Bome : 
generated such that each element of the model was represented by at least tro eme, noar 
forms were developed of sixty items each: Student Perception, Student Choice, Teacher Chole 
and Principal Choice. 


(b) Interest in the science course 
———s-h the Science course 


An important index of total class environment during implementation of a ees gunvipulnm 
is that of student interest in the course of study. The general form developed by BHance ana 
Remmers” has been widely used in a science-adapted form. Hedley” developed an inventory 


T 
Mayer, Unpublished Evaluation ... op. cit., p. 168. 


2 
ibid., p. 170. 


3 

Rolad B. Berty, "A Study of the Relationships Between Classroom Activities, Student-Teacher 
Relationships and the Characteristics of In-Service Secondary School Science Teachers of Costa 
Rica". Unpublished doctoral dissertation, The Ohio State University, Columbus, 1975. 


4 s 
Edwin L. Shay, "A Study of Relationships Among Selected Teacher Variables and Expressed 
Preferences for Student-Centered, Non-Direct Science Education", Unpublished doctoral 
dissertation, The Ohio State University, Columbus, 1974. 


5 " 
E.B. Silance and H.H. Remmers, "An Experimental Generalized Master Scale: A scale to 


measure attitude toward any school subject". Purdue University Studies in Higher Education, 


Vol. 16, p. 84-88, 1934. 


f Mayer, Unpublished Evaluation ... op. oit., p. 131. 
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consisting of seventy-two statements with a Likert-type response format to assess student 


acceptance of course material, course content, laboratory work, interest in the course, 
involvement, and satisfaction of perceived needs. Reliability and validity estimates were not 
reported in its use with tenth-grade Manitoba students taking the "General Course" science 
programme. More recently, the semantic differential format has been widely used, such as in 
Henson's study! of science achievement of students using the Earth Science Curriculum Project 
materials in Oklahoma City schools. His scale attempted to measure attitudes of students 
toward school, science, learning earth science, reading earth science, earth science experiments, 
earth science teacher and earth science classmates. His semantic differential used those seven 


concepts, each with nine adjective pairs as the response scale. 


(c) Teacher variables 
he nature of the implementation effort consists of the 
background knowledge of the teacher and teacher attitudes and opinions concerning appropriate 
instructional objectives. Unless these factors are consistent with the objectives of the curriculum 
being implemented, it is doubtful that an optimal classroom climate can be achieved. Mayer, 

s the achievement of teachers enroled in the summer 


et al?, developed an instrument to asses: j 
Phase of a workshop designed to prepare them to teach the Earth Science Curriculum Project 
(ESCP) materials. Their Earth Science Test. was prepared using as a reference the 
behavioural objectives as stated in the "Teachers Manual" for Investigating the Earth, the ESCP 
Student textbook. 


Another important dimension of t 


Merkle designed? an instrument to assist in determining the effectiveness of a leadership 
Workshop on elementary science. it was designed to test knowledge of the programme 
Characteristics of AAAS Science - A Process Approach and of Science Curriculum Improvement 
Study. h d 
5 i teachers, used a semantic 
Wonkka? , in evaluating i dud seep i dies 
differential of twenty-five nooner and sixteen pipolar adjective pairs to determine teacher 
attitudes toward the "new" curricula. James? , in working with student teachers in science, 
used a semanti em ntial of twelve concepts and fifteen bipolar adjective scales to assess 
attitudes Cee run P once. Validity was established through use of a factor analysis of 
results, 6 to b d with und duate: 
: le to be used with undergra uates in teacher 
Schwiri Science Support 889 * atti s 
iiie niet yore teachers to assess their attitudes toward science. Items were 
nd elementary rd Barber's contentions regarding the nature of 


Selected “a accordance with Bern? > 
Science Lelea a the growth and the development of science in a society. 


è i dary schools of Costa Rica, developed the 
B i : i education in the secon à JORN , 
Rati etd in studying pr, ras Education to determine what objectives teachers were 
epp crede of c EU Stats objectives from the ministries of education of five Latin 
ing. e revie 


n-service institute 


 -— 
ibid., p. 134. 
L. White, "Evaluation of an In-Service Program 


and Arthur 
Vol. 59, p. 145-153, 1975. 


Victor J singer’ 
. Mayer, John F. Disinger: ; 
for Earth Science Teachers" Science Education, 


op. oii, p. 175 


3 
Mayer, Unpublished Evaluation --- 
ibid., p. 178 


5. 
ibid., p, 172, 
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Berty, op. cit. 
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American countries, identifying forty-four objectives. ^ Respondents rated each on a four-interval 
scale from "Very much emphasis" to " Very little emphasis". 


CONCLUSIONS AND RECOMME NDA TIONS 


The type of data-collecting procedures discussed in detail in this paper is only one of several 
formal and informal types that have been used in curriculum evaluation studies. The reader is 
warned, therefore, not to presume that the formal data-collecting procedures discussed herein 
are the best for use in any situation, or that instruments developed to assess a student behaviour 
resulting from one situation or curriculum can be used intact with another situation or with 
another curriculum. 

In formative evaluation, most workers have found interviews, classroom observations, open 
response forms and other informal methods to be more useful in providing the detailed information 
necessary for effective curriculum revision. ^ However, information on student characteristics 
which should be used in making decisions in formative evaluation can perhaps best be obtained 
through the extensive use of the types of instruments reviewed in that category in this paper. — 
The types of procedures described here are particularly useful in summative stages of evaluation 
where there is some attempt to compare the results of one curriculum with those of another. 

For this purpose, a higher degree of objectivity of data and data collected from a much larger 
sample or population than is possible with informal means, are desirable. A widespread testing 
of the curriculum in a wide variety of contexts and the provision of evaluative data inom md 
testing can provide essential information for decision-makers in selecting curricula or curriculum 
elements for implementation. This type of data can probably best be supplied through the use of 
objective, respondent-completed instruments. bic. 

A final warning: the reader should not take the description of ease ee 
assessing student performance and climate or process variables as an endorsement of € lis 
instruments for use intact elsewhere. For most purposes, an instrument must e errem 
measure the specifically-intended outcomes of a curriculum and the procedures and philosophy 
of that curriculum. These instruments are described as models that could be used by an ized 
evaluator to develop similar instruments for his or her own purposes. It should be emo 
however, that instrument development is not a simple procedure. To develop sufficiently Si $ 
valid, and objective instruments requires heavy investment of time and creative energy. 3 ts 
sufficient effort and expertise are applied to instrument development, the information yielded, uy 
their use may be meaningless, or worse, misleading. 

At least in one area of instrument development, it might be useful to redirect 0 ; 
We are at a point where there is a wide range of attitude instruments available, most purporting 
to assess similar aspects of student attitude toward science and scientists. Effort should now 7 
directed toward the revision and/or refinement of these instruments along the following guidelines: 


This construct 


ur efforts. 


The specification of a clear theoretical construct to underlie the instrument. 
should then guide the selection and development of items. 


Avoidance of confusion between different theoretical constructs. If more than one is to be 
included in a single instrument, each should be identified and scored as a separate factor. 


Provision in the design for filtering out influences of respondent knowledge about the 
Scientific enterprise from attitudes towards it. 


The elimination of defective items, such as those that combine two or more different 
understandings or perceptions. 


The preliminary trial of the instrument on a population with characteristics approximating 
those of the population with which the instrument is to be used. 


The refinement of the instrument for each factor or subscale such that a reasonable internal 
consistency is obtained. (This relates closely to the first two items.) 
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Determination of the stability of the instrument through a test-retest technique. 


The use of factor analysis to empirically validate factors. 


A few attitude instruments developed and refined in this manner will prove much more useful than 
the continued generation of unrelated instruments in isolated studies. 
The assessment of gains in science concepts and processes and in knowledge about the 
Scientific enterprise will, and probably should, continue to be the prime focus of evaluation efforts. 
es will be highly dependent upon the particular 


Since the content of integrated science cours 
Curriculum, in that it must reflect local conditions of culture and environment, it will be fruitless 


to attempt to develop instruments that can be used extensively. Instead, instruments have to be 
Assembled for the specific curriculum in question. The development, testing and refinement of 
items, however, is a difficult and expensive process. Also, even though instruments must be 
Specific to the local curriculum, individual items would have wide evaluation utility for curricula 
throughout the world. Therefore it is recommended that item-banks be developed and made 
available through an international agency such as Unesco. Past evaluation efforts have produced 
an extensive variety of items. These items should be collected and classified according to some 


Conceptual framework. One that might be adapted for this use is that developed by the 
It consists of four categories: the unifying concepts 


Federati ifi i cation ~. 

of rae a Bus, prassi of science and values of science. Extensive effort 
has gone into the development of ite n the areas of biology and physical science under 
the auspices of the Australian Council for Educational Research and the Curriculum Centre of the 
Education Department of Tasmania. Such an item-bank could eventually provide standardized 
items on a world-wide basis on the major common aspects of science. . It would not preclude 
the local development of additional items relating to instructional objectives that may be peculiar 
to the local situation because of cultural and environmental factors. The availability of such 


à i f achievement testing: use of 
an item- ili t has long been the ideal o! 
lees aang m ben items are randomly selected from a large bank (domain) that 


has : ll the objectives of the curriculum being evaluated. In the 
reper sm vi E Mus of the amount of e im ber cepi oe to M 2 
domain of a thousand or more items required for an -— en = es : s reap Miri x 

Most of the available instruments have been develope E P : Ae ; eae a 
As we have b ore sophisticated in our efforts at m um. i gg w PETR 
teailzo that careiails need to be higit Herbi D onori for mo LEAD aoe pares 
Which are relat in effective learning- To develop such omg a es ehe E ante 
to identity tomos characteristics and to be able to measure them e 


r : i ts. Efforts such as that by Tisher and Dale, in 
elative effectiveness of curriculum a Piagetian stage of development relative to physical 


de f ifyi 
sie a instrumentation for Ideni needs to be extended into other science concepts. 
tfort concepts, are encourag rS ntification of out-of-school experiences which relate to science 
aski must be directed toward i aa instruments accordingly. Related to this is the effect of 
evement and to the developmen 2 comment on this problem relative to the 


» t 

Pius on science learning. Griffiths = poe for Papua New Guinean students. Their 

Velopment of an interdisciplinary oe aa scientific concepts and much practical experience 
ar 


e š ; 
iP erience demonstrated that there fence courses which were not necessarily present in the 

a Plicitly assumed in most western © other hand, certain scientific modes of thought, particularly 
ui ckground of their students- On highly developed in students in certain non westenn cultures. 
assification, have been found tO be loping science curricula within highly 


: t in deve 
uch concerns become especially importan 
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pluralistic societies, and in such situations at least, it would seem important to have instruments 
which would identify student differences arising from cultural factors. There are no doubt other 
areas of student background characteristics that must be considered as well, such as critical- 
thinking behaviour in science. This is where the future emphasis upon instrument development 
should occur. 

It is encouraging to see the development and use of instruments relating to classroom climate 
variables as they occur during the implementation effort. The work of Kochendorfer and Sagness 
in the United States and Best in Australia, using various forms of a Classroom Activity Checklist, 
and Walberg and Anderson in the United States with the Learning Environment Inventory | , have 
been pioneering efforts in science education. Their efforts, and those of others using their 
instruments, have demonstrated that the student is a highly reliable observer of classroom 
practices. Future instrument development should capitalize on this fact and extend their work 
to a wide variety of curriculum-dependent, learning-environment characteristics. The collection 
of this type of information is extremely important in judging why a particular implementation 
effort was either successful or unsuccessful and in describing for the decision-maker the climate 
in which a given curriculum can achieve success. 

The case study by Sim of the evaluation of integrated science teaching in Malaysia includes 
the description of a broad-based data collecting procedure containing many of the elements 
recommended in these conclusions. The reader is referred to Chapter 15 for a description of 
the types of variables that are being measured and the data gathering techniques that are being 
used in that study. 
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T Assessing student progress in integrated 
science using teacher-prepared methods 
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SUMMARY 
cience using teacher-prepared methods, the 


In this paper on assessing student progress in $ 
The first is the conceptualization of assessment 


emphasis has been placed in two main areas. 

in terms intended to be useful to the teacher. 

Of students as much more than pencil-and-pape 
teachers may explore. 


r tests and the outlining of a few areas which 


The nature of assessment 
courses and statements made about how they should 


content of the assessment instruments reflect what 
ately for students. Their real science 


Despite the noble aims of integrated science 
be taught, the manner of assessment and the 


the actual curri i d to be about most accur 
culum is suppose 1 
curriculum is defined by what is assessed, because they know that, based on this assessment, 


they either pass or fail, and their future is in the balance. For this reason, one can see the 
importance, in the framework of integrated science curriculum, of the way the students in the 
COurse are assessed by the teacher. 


At this point, we should define 


assume everyone knows what it means, OF to slide f 

‘evaluation! "adinim the two to be synonymous. We may also describe a student's 

'performan o! ph ticular test and then draw 'inferences! from such performance about the 

learni LUNES iae od t We may establish toriteria' for passing or failing a student, or 

for V ilities en i ent. may have to 'report! to the student, the headmaster 
ing a credit rating. 


Eventually, we ] 
lin ults of our assessment. 
Bons Amd. inis pede e been found useful by groups of science teachers in South 
E . 
Australia for assessment discussion may prove helpful: 


vity which may be observed or reacted to; 


It is easy to use the word assessment and 


our terms. 
from the word 'assessment' to the word 


performance - is any acti 
infer is any statement about the unknown based on the known; 
ence - 
Judg ; fe: pny statement of approval or disapproval which generally closes 
emen - is ; 
further thinking; 
im rformance and may be descriptive of fact 
a = i description of a pertor à > 
Ssessment ereraa or judgemental in emphasis; 
stated aspects of performance on which an assessment is based; 
criteria - are 
reporti is the process of communicating an assessment. 
T ang = 
tricted definition to be a statement about the 


onsidered in res 


criteria, and so all evaluations are assessments. 
, 


5 The term ‘evaluation’ may be © 
elationship between performance and 


The second is the clear identification of assessment 


, 
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An important role of the science teacher is to attempt to find out what students are thinking 
and how they are learning from a wide range of external performance cues shown by them. 
An example of such a cue may be a simple exchange in the middle of a lesson, such as: 


performance: A student interrupts the teacher in mid-sentence with 
a loud noise. 


inference (by teacher): The student is testing me. 
judgement (by teacher): The student is rude and should be punished. 
Such interchanges have profound effects on classroom learning. A second example: 


performance: A student interrupts and asks a teacher why a person 
can drink vinegar if it is an acid. 


inference (by teacher): The student has not used information he should have 
on strong and weak acids and dilution. 


From the inference the teacher may either make a judgement and hence closing thinking or 
decide a course of action. For example: 


judgement (by teacher): The student has not been listening in class (producing 
closure). 


course of action: The student needs some help in understanding acids 
(options remain open). 


These sequences of 
Student performance 
teacher inference 
teacher judgement 


or Student performance 
teacher inference : 
Student and teacher's course of future action 


i t 
are common in a classroom. They may arise from verbal interchange or from the student's 


performance on a test. 

Assessments are usually of at least three different kinds. There are (a) descriptive, 
factual summaries; for example, on a test a student could match only three scientific terms 
correctly from a group of ten; (b) inferences drawn from the descriptive summary; for 
example, having achieved only three correct matching of terms, the student has not learned the 
terms. This leads to the action of advising the student to spend more time learning gh oa 
and (c) judgements made on the basis of the inferences; for example, the student was too lazy 
to learn for the test. ^ Although the assessment of performance is often a complex intuitive 
business, it may be helpful to consider it in stages: 


observation of the performance 


Y 


factual summary 


Y 


inference 


judgement course of action 
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When being assessed by a teacher, students sometimes do not appreciate the distinction 
between these stages or types of assessment. In fact, the inferences they may draw from the 
facts of tlie performance, or the judgement they may make, may be quite different from those 


made by the teacher. 


From the teacher's perspective, the set of events may be described as follows: 


observation of performance 


Student achieved three correct out of ten. 


(agreement by student and teacher): 


teacher inference: 


teacher judgement: 


From the student's perspective, 


student inference: 


student judgement: 


If the student's inference and 
agreed course of future action by 


however, there are widely divergent inferences, 
then likely future c 


the student as in the example, 
difficult to achieve. Any future 


as an authority figure for there to 


in the student's level of learning. 


their inferences and assessments, 
When they do not, then co-operation 
need to put the student under duress to 


Criteria for assessment 


Much of what has been stated so far h 
Or the students about their achievemen 


State what we expect a performance to b 
formance on actual 


hecklist that may 


] Criteria for assessing per 
Criteria may be provided by 4 € 
by the teacher to assess (say) W 
of such criteria may be for 
a certain number of words, 
Contain diagrams of five different 
teacher know what is expected. 
May be made on any factors wh 

The relationship between ac 
three kinds: 


comparative 


diagnostic 


hether the stu 
the teacher to esta 
will identify the na 


ich may be consi 
tual performance 


The student needs to spend more time learning. 
The student is lazy. 
the set of events may be described as follows: 


The teacher did not explain the terms well 
enough for me to understand. 


He is a bad teacher and I can never learn 
with him. 


judgement happen to be the same as the teacher's, then an 
student and teacher is likely to have some success. If, 
drawn either consciously or unconsciously by 
ourses of action are going to be much more 
course of action will require the teacher to use positional power 
be much hope of the course of action producing an improvement 
When student and teacher agree either overtly or covertly in 
co-operation in future between student and teacher is possible. 
becomes much more difficult and the teacher is likely to 


force learning. 


as been based on intuitive understandings of the teacher 
tasa result of the learning process. If we openly 

e before we observe it, then we are establishing criteria. 
science learning may be specified. Such 

be used by the student, by another student, or 
dent can use a balance properly. Another example 
blish that a report of a field experience shall have 
tural habitat of five birds seen in the forest and 
When criteria are stated, then both students and 
ja are not achieved by all students, then judgement 
dered to affect the achieving of the criteria. 

and the established criteria may be of at least 


leaf forms. 
]f these criter 


the teacher Says 'this student's report is well 
for the class', or 'your behaviour in 
s much worse than the other classes 


for example, 
below the average 
practical classes i 
I teach’; 
e, the teacher says ‘this report does xot 
tain a summary or conclusion about the kinds of beaks 
2d the birds in that part of the forest have', or ithe Bad 
Lodi of the class seems to stem from you three 
working together. I think we will separate you; 


for exampl 
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predictive = for example, the teacher says 'if you do not learn to put 
in summaries and conclusions in your reports then it is 
unlikely, if you ever become a scientist, that your work 
will be useful to others', or 'if this class continues with 
this bad behaviour, then many of you will fail at the end 
of the year!. 


The function of assessment 


Generally in classrooms, it is assumed that the comparative, diagnostic and predictive assessments 
are made by the teacher. ^ Usually this is so. Yet if the students are prevented from being 
involved in these processes in the classroom, they will make their own assessment themselves, 
either covertly in class or with their peers outside class time. Since one of the generally-stated 
aims for integrated science is to foster a student's decision-making ability, it seems important 
that students themselves as well as the teacher, be actively involved in the evaluation process. 

We can assume that most students have hopes and aspirations for themselves which lead them 
to want to know: whether they are capable of and are suited to what they want to do; whether 
what they are doing is worthwhile, either in the present context of schooling or in the future 
context of employment; how to improve what they are doing so that they can do it as well as 
they are able; and how well what they are doing compares with how well others do the same or à 
similar thing; or, does their performance meet some minimum acceptable criteria for the task 
or activity. 

The first of these assumptions relates to our providing better information for students about 
the science they are capable of learning, and indirectly to the teacher about teaching. pope 
Science currently seems to be what teachers of integrated science want to, or are given to teach. 
How do we decide what is appropriate for the group of students in the class, given the resources 
available? The criteria for deciding this information ought, if possible, be agreed upon by 
students and teacher, either by a quite formal process (especially in the senior school) or by a 
much less formal approach (in the lower grades). It ought to be possible to help students 
understand how the particular topic chosen suits their present interests and level of knowledge, 
Skill and ability. 

The second assumption relates to the worth of what we have the students do. If the students 
value what they are learning, then they are more likely to want to learn it. If the teacher can 
communicate to the students why a particular topic or set of learning experiences is valued, then 


the students are more likely to value it too. Evaluation is about values, both of the teacher and 
of the student. The very process of evaluation has its effect on the development of pope itis 


attitudes to science and to learning. And the way the teacher assesses and judges will rev 
teachers! own value system to the students. 

The third assumption relates to improvement. When criteria have been established and 
agreed upon by student and teacher, evaluation lets the students know how far they are from als: 
meeting these criteria. Assessment against overt criteria informs students at one of three len 


1. That they do not know a particular piece of information or cannot do a particular task; 
the pass-fail kind of information. 
2. What they do not know, or which tasks they cannot do; such information identifies proble 


For example, 


areas, but does i ement. 
not help the students know how to go about improv a. but you 


the teacher states, 'your knowledge of the structure of the human heart was soun 
did not understand the mechanism by which the heart continues to beat'. 


3. What they can do to improve what they do not know; here the emphasis is on in oce 
and remedy with the assessment being part of the instructional process. To pag 
second example, specific pages of a book may be indicated to the student or he may De 
asked to follow the matter up individually with the teacher later. 


" ind of 
We are confronted with a hierarchy of assessment at three different levels. The first k 
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information can be set against criteria, some of which may be unknown to the students. Public 
examinations are an example of this kind. If students do not know the relevant criteria, then they 
cannot make inferences which will help them take a course of action to improve. The final grade 
or mark given in a public examination is a judgement and produces closure. The second kind of 
information provides some information about the criteria and is more open about the teacher's 
evaluation against criteria; but if this is not to produce closure, then future action is largely up 
to the student. The third kind of information provides the criteria, how close the students come 
to meeting the criteria and what the student needs to do to meet the criteria eventually. 

The fourth assumption relates to the genuine desire of students to know how they compare in 
their learning with others. Information of this kind is provided by ranking students on a particular 


Set of tests. From the distribution of marks and the student's rank, we can assign grades. The 
to some 'average' student in the class. Such 


comparisons can be important to the students, their parents and the teacher, but can reinforce 
failure for those students who are always below average. The challenge for the teacher is to 
establish the appropriate balance between comparative criteria for all students and absolute 


Criteria for each individual student. ps d , 
Our tests in the classroom should help us as teachers to obtain information about our students, 
and help the students to obtain information about themselves so that the teacher's evaluation of the 


results and that of the students lead to judgements or courses of action from both which are as 
congruent as possible Tests should be looked on as aids the teacher has for helping students 
make decisions about themselves, 25 much as providing opportunities for teachers to make 


judgements about students. 


The natu 
re of a test 
nce of students. It may sample knowledge, 


mple. The more representative the test is 
accurately will the test sample reflect the total learning 


of each student Of course, the teacher's perception of how accurately the test samples expected 
. c , 


learning mas her quite different from the student's perception. The more open the criteria for 


ass 1 the more likely are the student's and the teacher's 
P ement qf periozmaace Ce pcr dent, it is what is sampled that provides the evidence 


pe i aie 

ace eee pee the curriculum. Ib s ee s = amne er 
to ask a class to set a test on a section of work, perhaps io onena i Festa = hs punted 
lack of sophistication in test setting, what is obtained does show ry y ents in 
the nt Purse €: Ashe eo teacher from a particular ope Raced ion Era into 
Account; the sedie id course; the special interests s the | ed aoe me r ? aeui] of 
"s group of students; the time available for the RN it as ern nd in mde "i 
Pre-requisites for the next section of learning; the 7 Í 


e i 
Xpression of the ideas. cribe any device or method used to sample students' 


The word 'test! may be used nn pen-and-paper kind directed at the individual child. 


behaviour and may or may not be 9 


the performa 


A test provides a method of sampling 
more than a sà 


Skills, attitudes; but it can never be 


9f the learnings expected, then the more 


Main uses of a test 
h measures pre-requisite knowledge and skills that a 


uence. Such tests are useful in preparative evaluation, 
sed tudents' abilities, interests and knowledge prior to 
stablishes what is expected as entry criteria and then 
of students meets these criteria. Should the 
truction may be given to those students falling short 
rticular instructional sequence may require the 


A 1 
test may be used as a pre-test whic 


Student has before beginning à learning fs 
here the teacher makes an assessment ^ e 

© beginning of instruction. The lage 
i to see how closely the particular gro ae 
á "dents not meet the criteria; prelio a pa 
f the anticipated behaviour. FOF example, 
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student to use a microscope. A simple pre-test would show those who meet criteria for success 
in using it and those who do not. Those who cannot may then be given special instruction in the 
use of a microscope. . 

Tests are used to help the teacher assess how effectively student learning is occurring during 
the instructional sequence. This may be done by checking simple homework questions, asking 
students what difficulties they are having with the work or watching to see whether the students 
are undertaking purposeful activity in the laboratory. This is very much a matter of experience 
and leans heavily on informal monitoring by the teacher of what is going on. Often the difference 
between a 'good' teacher and a 'not so good' teacher is in this constant process of intuitive 
monitoring, and it is an area in which much more help ought to be given to teachers. 

If the teacher senses that something is specifically wrong in the instructional process, then 
diagnostic tests may be used in an attempt to focus on the difficulty. Teachers can quickly 
develop simple diagnostic tests. They should be brief, direct, and in such a form that, 
wherever possible, the students in discussion with the teacher can diagnose their own problems. 
Diagnostic tests are used for checking simple’ skills; for example, using a balance, pipette, " 
Scales of various kinds, slide rules or log books: for checking basic knowledge such an pyon S, 
formulae and equations, or terminology, say the parts of the body, or simple demonstrations o 
the understanding of basic laws. Often, in-depth interviews with students about their work can 
provide diagnostic information which is superior to that obtained in other ways. : 

Finally, tests are used to assess what students have learned in the total instructional " 
programme. These tests are generally used to compare student levels of attainment € $ 
other, to provide rank scores and who is to be 'top of the class' in the particular test. ; gd 
tests may be provided by external examiners, or as separate test batteries which ane Piae vat 
integrated science materials being used. | Knowledge can be tested quickly if so-calle 9 iid ] 
multiple-choice, pen-and-paper tests are used. Reports of work done, and discursive an B 
to specific questions are also used. Much more rarely oral reports either to the class od a 
the teacher, or in-depth discussions are used to make evaluations. Criteria for n: 

Such non-written methods are much more difficult to state, but they can give à better tota — 
picture of learning than pencil-and-paper tests, particularly when outcomes other than knowledg 
are expected. 


Criterion- and norm-referenced tests 


There are two basic kinds of tests which teachers can prepare. The first are oa 
tests where student performance is measured against some pre-determined standard of re enr 
Such standards for an individual student do not directly relate to the performance of pma : ee 
but, of course, must be based on a realistic expectation of what may be achieved. ps uiu 
Sometimes called mastery tests, are used generally as pre-tests or formative tests nt au 
tests. They may also be used as summative tests, but it is usually at the end of a lea unm 
Sequence that comparative data are required. With such tests, the students would know coven, 
they would have to be able to identify, say, eight species of plant out of ten ‘aaa d Ee p en 
before it would be assumed that sufficient "mastery! of a topic on identifying plants ane we than 
achieved. In such circumstances, some students would achieve mastery much more dum y 
others, and hence their use is often based on a more individualized approach to aen ET 
Teachers should be encouraged to develop this kind of test more often. They sta eie 
for the student the performance criteria expected and orient the student toward success, i 
than failure. 


rmance 
The second general type of tests are norm-referenced tests. These relate ips e^ 
of a student to that of other students in a class, or other clearly defined group. d with 
re 


acceptable depends on the relative performances of one group of students as compar rtant 
another. They tend to use competition as a major motivating factor. They are ene ferenced 
making performance comparisons, for example, to show whether a particular criterion-re e 

test is too severe. 


for 


84 


Assessing student progress in integrated science using teacher-prepared methods 


Different contexts for assessment 


Although two very specific types of tests have been given, the concept of assessment in an 
integrated science classroom must have a much wider perspective than simply obtaining information 
from pencil-and-paper tests. Teachers are encouraged to explore the different contexts in which 
assessment may be made and the different media which may be used to vary the presentation of 
assessment material. Contexts which may be explored are groupings, places, time and structure. 


For grouping, we may assess individual students, small groups acting as teams, say in 


discussion sessions or practical sessions in the laboratory, and class-size groups where we may 


Wish to compare one class with another or simply determine how the class is progressing. One 
technique which gives a considerable amount of information about a class group within a reasonable 
test duration is to divide the class randomly, say into four groups. Give each of the random 
groups a different test. From this arrangement, it is possible to obtain four times the 
information about the class in the same amount of time. It does not, of course, give the same 
Specific information about each individual student in the class. 

Assessments may occur in many places, not just in the classroom. There is the laboratory 
for tests of practical skill and problem-solving. In the field, on field excursions, students may 
be expected to identify certain animals or plants; to sketch particular geological formations or 
to look for examples of man's impact on a particular environment. Evaluative discussions ought 
to form an important part of any field exercise. , . . 

. The home, too, can provide à place for assessment. Parents, if possible, may be involved 
in what is done at home. Students may be given checklists and diagnostic tests which may be 
taken at home. In many settings, however, very little can be achieved in the home environment. 

The library should be the place where information search skills are tested. For example, 
Whether a student can use a library card index, the index of a book, a data book of information 
about the elements can all be tested in the library. Some dente of this kind have been simulated 
in pencil-and-paper form, for example, by Fraserl, but it is by far the best for the students 
to be taken to the actual library. 

a The school grounds and neighbour: 
meo ing Students may be given ur 
Jg a a pene needs to be assessed carefully. In general, students 

Should enjoy the experience aS much as any other set of learning experiences put before them. 
his is much less likely in the competitive atmosphere of normative testing than in explicit 
Criterion-referenced assessment. Generally, too, teachers ep sey mie can ho T 
Y a student in a given time, particularly when ie Dia Teguined to; read and COPD ELE 
Y ents which give information at the 


Particular R 5 ; ount. Summative assessm 
question is taken into accoun! 
end of a particular instructional sequence may be weekly to annually, or at whatever seem 


ld be initiated as the need arises. 
Conveni s of assessment show e 3 
eni intervals. pes 2 Js important aspect to be assessed. How much time is lost 
at the b use of time by studen How long do particular activities take? When should a discussion 
be shut b cbe a eerie 4 aspects of student use of time which should be monitored wherever 
ese are 


Possi 4 dri 

ee against criteria. thought to the structure and form of their test. Highly structured 

uno posed should give some d objective type, allow specific identifiable performances to be 

d Or example, the X oua K a in number form. For this purpose, we can use multiple 
ed and the result of the test P little flexibility as to which answer is 


choi HA es which give 
vice Or matching pair or tre tae WE correct answer is among the possible responses and 


Corre ts 

tens What the teacher expen stanes different response of some students. More open 

teen = little room for the PAPP ncourage d by having the assessment less tightly structured; 
ses from students may be to discuss a question like ‘What difference does the 


for example, students in a group 47° invited 


interesting and different context for 


hood also provide an 
d assess whether they can apply what 


checklists which woul 
] situation. 


1 


B " Monash University, 1974. 
-J. Fraser, Tests of enquiry § 


kills, Books 1-3, 
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use of fertilizer make to agricultural production?'. While certain responses may be predictable, 
others may not. It is in such more open situations that student attitude development may be 
assessed. In the question given, if the integrated science course had as one of its aims to 
'make students aware of the impact of man on the natural environment, we would hope to see 


Students raising issues related to human management of the environment. 

Much more weight should be given in classrooms and laboratories to observations of unplanned 
student performance, especially for monitoring the development of attitudes. Teachers should 
keep a 'daily log! of important behaviours of students which may relate to teaching objectives. 
Such anecdotal records are extremely valuable in helping assess the instructional process and in 
providing formative evaluation data. Specific difficulties students have with their learning might 
be listed; records of tasks which required too high a level of skill will be invaluable for the 
teacher in deciding what students might attain. 


The medium of assessment 


When considering the medium or vehicle of assessment we should recall that student responses may 
be of three kinds: written or spoken, or an action not requiring words. Thus in deciding whether 
a student can prepare a salt from hydrochloric acid and sodium hydroxide, we may ask for written 
evidence or an oral report to the class, or we may ask, given the two solutions, to prepare a 

dry sample of the salt. If we look at criteria which may pertain in the three cases, we can see 

that the three different types of performances allow us to assess different outcomes using the 

Same piece of content. These may be summarized as in Table 9. 


Table 9. Assessment of different outcomes using different assessment methods. 


C ee ME APR 


Written Spoken Action 
SEQ REPE icio ERR RR eire 
Individual activity Individual or group activity Individual or group activity 
States how Says how Shows how 
Written expression skill Verbal expression skill Manipulation skill paramount 
paramount paramount 
Interaction between individual Interaction between individual ^ Interaction between individual 
and paper and peers and teacher and materials 
Feedback re success or Feedback re success or Feedback may be immediate 
failure delayed failure immediate or delayed 
Recall from remembered Recall from remembered Recall may be cued by 
events important - cueing events important - cueing concrete objects and by 
by teacher difficult by teacher easy teacher 


ia isle RR cadet a 


on 4 
Whether one expects a written, spoken, or action performance from the student depends 


number of factors. These include: 


feasibility - sometimes only written or spoken responses are possible 


because of resources. 


Student age and - in general, more highly verbal responses are suited to 


stage of development highly verbal children. Younger children are better su 
to 'action' tests. 


ited 
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outcome being - if we want 
to know whether students reall 
tested action is the only way. UE SUM S 


tim i 
e available - spoken report can take five minutes, a written report fifte 
minutes and an actual demonstration a whole lesson d 


t all three methods should be used in assessment. Some examples 


The important thing is tha 
teachers are encouraged to explore the best 


idi types of student performance are given: 
or the particular task. 
Examples of written tasks include: 
use of pen or pencil and paper to report or respond; 
overhead transparencies prepared by student; 
open book or closed book tests which may be - 
multiple choice 
short answer 
true/false 
matching pair; 
essay topics which may be prepared or unseen; 
written reports of student stating 
what has been learnt 
what can be done which could not be done before 
what is found difficult 
what is found easy 
what is found interesting 
what is found boring; 


practical records; 
student laboratory prac 
student projects; 
drawings and sketches; 
report of a significant (to the student) investigation io be ee SY 


tical records; 


the teacher 
other students 
parents; 
to situations shown in O 
television programme 
. film or film strip 
paper report. 


written responses r main ideas from - 


be guided by fairly closely stated criteria; 

f laboratory work or student projects. Other tasks are more 
sum of the responses of the class to assess how effective 
on asking the students what they find difficult the teacher 


fi 
we more than fifty per cent identify valency and formulae, more class teaching time may be 
aside for it. If only one or two do, then individual work can be provided for the one or two 


a 1 
nd the teacher can go safely to balancing equations. 
Examples of spoken tasks include: 


For some of these tasks, the student needs to 


this ; 

etis particularly true of reports © 

ium where the teacher uses the 
ing has been. For example, if 


for example - 


oral answers to written questions , 
student explains to class an answer to a question 
at end of chapter, 
student answers to class a question selected at random 
from 4 box of questions; 


tions ~- 
ked by teacher 
d by other students; 


oral answers to oral ques 
as 


aske! 
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discussion/interview - 
teacher with individual student 
teacher with small group of students 
student with student(s) and teacher observer; 
audio and video tape - 
made by student and discussed by students 
made by teacher and discussed by students 
from outside source; 
reports to class - 
of an experiment completed 
of library search attempted 
of a field excursion 
of some unusual phenomenon; 
questions asked in class for oral response - 
by teacher of students 
by students of teacher 
by students of students 
(the first of these is very common, the second 
common and the third rare); 
discussion on - E 
written projects 
practical situations 
television or film programmes. 


Criteria for assessing spoken performance are less well identified than for written performance 
responses and for this reason spoken performance is seldom evaluated. It is interesting 
Occasionally to tape record a small group or class session to see which students in the group: 
Show they have grasped the content; can articulate ideas on the topic; ask good questions; 
contribute to the cohesiveness of the discussion; and create disruptive events. " 

If such tapes are made at regular intervals, some idea of class development of oral skills 
may be gauged. Simple checklists in class to note how many responses a day a particular 
Student makes and to what effect, are also helpful. 

Some examples of non-verbal action performances of students include: 

Students undertaking laboratory activities; 
students showing other students how to do something; 
Students manipulating equipment including - 
laboratory equipment 
video equipment 
audio tape recorders 
film - movie and still camera and projector; 
Students sketching - 
on field trips 
in laboratory; 
Students attending regularly and punctually; 
playing simulation games; 
practical tests of laboratory performance; 
students showing particular behaviour patterns on field trips, for example, 
reflecting conservation attitudes. ` 


The distinctions between writing, speaking and doing are arbitrary and some of we fiagh 
assessment activities for students require some kind of amalgam of all three. Written €-— 
assignments are generally most easily converted into marks and hence into grades peas od 
tests (for example, the so-called objectives tests) are easily scored; and somehow as 
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we are more comfortable giving a Score out of ten to a written statement than we are to either 
its spoken or demonstrated counterpart. For example, we would be comfortable assessing the 
written description of how to make salt, but much less so to assess either the spoken description 
or the actual practical making of the salt. This is mainly a matter of practice on the part of 
the teacher and of being prepared to try to establish relevant criteria for non-written student 


performances against which we may evaluate our students. 


Methods of assessing performance 


In most integrated science curricula, we are looking to assess four broad kinds of performance of 
y be useful to explore the four briefly. The 


Students. They almost certainly overlap, and it ma} 
first broad area includes the whole range of methods which may be applied to test knowledge or 


Cognitive outcomes. Teachers in general are good at preparing these tests and there is 
Considerable help available. A good source is Bloom, particularly the chapter by Klopfer on 
Evaluation of Learning in Science - This source is particularly useful for developing objective 
tests where the test item is related to a specific objective. The final test may then be either 
norm-referenced or criterion-referenced depending on purpose. A simple reference which refers 
Specifically to criterion-referenced tests is Gronlund^. A D 
When preparing an instructional unit, the total sequence of "objective process of achieving 
Objectives — assessment of actual outcome compared with objectives", should be considered 
together. ^ Consider, for example, à topic on metals. A general aim for such a topic may be 


'to develop i J metals in use in the community, how they are 
edge of common A 
obtained at pn uld be aware of the importance of metals 


The students sho 
in the community and have some appreciat 


ion of the different applications of metals. Knowledge, 
skills and attitudes are all part of the general aim, 
mance and 


and as teachers, we should expect any 
Progress to be reflected in student perfo! hence be — te total programme of 
instruction to achieve this general aim may be broken down as in Table 10. 


Handbook on formative and summative 


adaus, (eds), 
G.F. M "Evaluation of learning in Science, 


1 
B.S Bl a 
+ Bloom, J.T. Hastings and fer 
op uation, New York, McGraw Hill, 1971; L.E. Klopfer, 
hapter 18 of ibid. 


enced tests for classroom instruction, New York, 


2 
N.E. Gronlund, Preparing criterion refer 


acmillan, 1973. 
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Table 10. 


Intention 


Specific objective 


Metals are found in the 
ground as ores which 
do not necessarily look 
metallic. The names 
of simple ores to be 
known. 


Chemical processes to 
extract metals from 
ores. 


Metals have played a 
significant part in 
the development of 
civilization. 


And so on. 


Analysis of a programme of instruction" . 


Process 


Activities 


Samples of ores are 
examined with hand 
lens. 


Reduce some oxides 
of metals on a 
match head. 


Read books on the 
relation between 
metal discoveries 
and community 
technology. 


1 


Assessment 
NW RENE Coo kl er TRIER noi) aic AC MR E t Le 


Specific testable 
performances 


Other observable 
performances 


Correctly name eight 
out of a group of ten 
metallic ores when 
handed samples. Be 
able to use words 
like ore, mineral, 
mining and extraction 
correctly in written 
sentences. 


Students describe to 
teacher which oxides 
would reduce. 


Group discussion of 
effect of metals on 
civilization. 


Bp pg seen saga a geen aA 


If we prepare a table of this kind, we can see which of our expected outcomes ma 


y be tested 


by pen-and-paper objective tests and which can best be tested using some simple observational 


techniques. 


If we take the following kinds of objectives seriously in our integrated science courses ^ 


evidence in support of claims; 


co-operation; 


rational explanations; 


outcomes than to test ways of observing what the students are doing. 
is intended that the students should learn or perform while learning. 


by evaluating the processes whereby learning is occurring. 
assessed as outcomes at the end of a sequence; 
the students are learning. 


Some examples 2 


attitudinal objectives are better ass bé 
of attitudes and student behaviours which may 


observed while learning are shown in Table 11, over page. 


observations repeated honestly; 
and concern with the consequences of technology - we are not looking 1 " 
They describe the ways ! 
So they are best assesse 
Cognitive and skill objectives may 


ess for 


essed while 


1 r 
Based on Australian Science Education Project, Metals From Their Ores, ASEP trial version, 


Melbourne, 1973. 


2 ‘ 
Based on material in Education Department of South Australia Secondary Science Curriculum 


Committee, Do-it-yourself Curriculum Guide for Junior Secondary Science (trial ed 


Adelaide, 1974. 
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Table 11. Examples of attitudes and student behaviours. 


Attitude Student behaviour consistent Student behaviour inconsistent 
with attitude with attitude 


Awareness of purpose of Can give outline of procedure Can only proceed if instructions 


experiments. for experiment relating to its are given. 
purpose. 

Respect for data. Records own observations, not Aaks what the result 'should' be. 
what he thinks ought to be 
observed. 


Records unexpected results. Records what 'ought' to be seen. 


Makes sweeping generalizations 


Accepts 'no conclusion’ as a k 
on inadequate evidence. 


Rational interpretation 


of results. possible outcome. 

Desire to understand Reluctant to use 2 formula Student finds out 'the formula’ 

basic principles. he does not understand. and uses it to get answer. 
Asks questions when this 


happens. 

Loses interest in experiment 
before complete. Finds out what 
happens from others. 


Persistence. Continues investigation until 
complete - tends to ignore 


end of lesson. Repeats 
parts of experiments 
'to be sure'. 


The teacher may extend these and also explore which teacher behaviours would encourage the 
Appropriate responses in students. 


in students may best be assessed by direct observational methods. 


the notion of 'enquiry' is dependent on the students having the skill 


lop skills indirectly. 
to undertak vars investigations. Oien We expect students to deve ly. 
Such skills phe qp nn ecified procedures; recording data accurately; using specific 
i 3 i i ference sources. 
Pieces of a inding information from rei . 
Man supe € i Eer science investigations mas" be tens icc pd poss 
Skills ibus 8 wee pees ray) oscilloscope; using 2 classification key; and using correct 


levels of a surement. d i 
s Of accuracy of menm i gna of oth Ka Ra Gu e im 
indirectly and those we must be prepared to guia RH s Ad eed data “inte an data 
Problems, designing experiments, ing data, recording an " tif a ifie iain, ic : 
amd sition é lg ing s vnd offi x be useful to help identify specific skills. 

or oa sone e and pr ocessing data may require -p ad i E able to: seee an 
aPpropriate ve posita n data; record data accurately in tabu : * ie ler : "n t 
mathemalteel ei T $ organize data; interpolate and extrapolate trom graphs; selec 

erations to re tables and so on to display data. 


" i i i . and use charts, graphs, 
PPropriate method of displaying iae to prepare such checklists. 


Teachers could work together in & 


ta d The development of skills 
1 integrated science courses, 


Reporting assessments to students 
s the reporting phase, where the evaluation of the 


The final phase of classroom assessment i 
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students' performance, against the established criteria for acceptable performance is made known 
to those interested in it. The form of report should vary according to who is to act on it, 
since no evaluation should be an end in itself, but rather give evidence concerning the selection 
of a course of future action. 

Generally, decisions about student progress might be considered as being made by either the 
teacher, the student, or the parent. The teacher's influence is profound. But decisions are 
made by other groups as well and the whole matter of decision-making has been taken up by 
Welch (Chapter 3). Assessment reports may be oral (most common) or in writing, and may be 
directed to the individual student, small groups of students, or class groups. Reports may be 
useful to the teacher, other teachers, employers and parents, or as feedback to those who designed 
the curriculum. The report phase is the culmination of assessment and needs as much care and 
consideration as the development of any instrument on which the report may be based. 

How does the teacher of integrated science cope with all these demands for assessment? 

Often there is enough difficulty in simply keeping up with the content of the course. The intention 
has been in this paper to show that assessment is a natural and integral part of the teaching process 
and must be treated by the teacher as such. The best teachers may be those whose intuitive 
assessment ability in a class situation is most highly developed. The first thing teachers are 

. encouraged to do is to recognize when they are making assessments in the classroom and how these 
assessments may lead to better decisions. One sure aid is to turn the teacher's intuitive 

implicit criteria into explicit criteria, agreed on by both student and teacher. 

The science teacher in the integrated science class may be described as first a planner, 
Second an organizer and third a leader. These three processes are likely to be ineffective if 
the teacher does not have an effective monitoring system concerning what is going on in the 
classroom. It is assessment that is the core of the monitoring process and must guide future 
planning, organizing and teaching. 

Too often we think of assessment as being test-generated numbers. Numbers are important. 
We do need to compare our students. But we ought to explore ways of putting numbers to 
assessments other than pencil-and-paper tests, so that the numbers which we use to determine 
the future of students relate to the whole spectrum of aims and not only those which can be SO 
assessed. 

Teachers often seem unwilling to involve each other in the instructional process. 
Sharing of ideas by one's peers can be a most helpful process. An idea crystallized by 
Wrigley! of a ‘critical friend' to discuss student assessments is one such idea. A person who 
will 'talk through' as an equal what is proposed by the teacher for the class gives the teacher ae 
Opportunity to reflect on what is intended. This is ‘reflective evaluation’ in which a few minutes 
of discussion with a critical friend may help clarify ideas or interpret incidents which otherwise 
might be lost. 


Mutual 


The role of science teacher associations and other similar organizations of peers in ‘ile 
a forum for teachers to discuss their ideas on assessment should be emphasized. Teachers ei 
their own educational context are the people in the best position to decide what may be — a 
in their own local area, and it is through the teachers themselves that change will occur. B 

are definite limitations to what individual teachers may achieve in their classrooms without t - 
benefit of pooled ideas. This is particularly true when teachers explore assessment outside 
usual paper-and-pencil kind. 

In conclusion, I wish to make some points about assessment in general. 
Schools which are still, or recently have been, dominated by public examinations will rea 
impersonal such examinations are. They have their place, but one tendency which they 
to make classroom assessment impersonal as well. Classroom assessment is a very perso” 
matter to the students and to the teacher - particularly the formative assessment and the 
assessment of attitudes and process skills. 


Those who are in 
lize how 


have is 


1 : 
J. Wrigley,'Fools and Angels', Schools Council, Dialogue, Vol. 15, 1973, p. 8-9. 
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bye science is a rubric for organizing science ideas in an educational way. It is 

d ona. because it involves students in learning through the interactions of student with student 

nd Student with teacher. The process of assessment is a process of discourse, of information 

ies and information receiving, of drawing inferences and of making decisions about people. 

ri one of the most sensitive activities in which a teacher will be involved. For this reason, 
eacher should not make assessments too instrument-centred. Rather, they should be 

person-centred, with as much emphasis upon the students being able to make their own evaluative 

judgements against mutually-agreed criteria as upon the judgements being made by the teacher. 
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8 Evaluating student progress in integrated 
science — public examinations 


John F. Kerr 
University of Leicester, United-Kingdom 


SUMMARY 


We shall be concerned in this chapter with ways of im 
discriminate between individuals but also to contribute to the mastery of those skills and concepts 
which the teacher and curriculum developer desires. The effects of public examinations on 
teaching and learning integrated science will be analysed, techniques of examining will be 


reviewed, problems identified and the way ahead considered. 


proving public examinations, not only to 


Although the form of public examinations is a perennial topic of discussion, their role in 
Previous chapters have stressed the need for evaluation in 


evaluation tends to be disregarded. 

iculum decisions for course developers, teachers and 
Students, Traditionally, public examinations attempt to measure individual attainment or 
achievement of students, t process W n or sorting for something, however 
egalitarian the particular society may 
ag of the Christian era, the soci 
M ality of opportunity and less sponsors 
condary and higher education systems 19 strong, i 
9f public examinations is reduced; but in most countries, 
ave been severely criticised for centuries, they continue to 


in practi 
It vx ede €— "iE aspect of evaluation - the measurement of student progress by 

Public examinations - should have received so little attention in the appraisal of new curricula 
and their diffusion into the school systems. In recent times, as indicated by some papers in 

is volume, the emphasis has turned towards the analysis of curriculum materials and the use 
of Observational elaine to identify styles of teaching and patterns of interaction between 
teachers and students Through these new techniques and insights, our control of curriculum 
Change jg beconit B e deliberate and effective. There is also strong evidence! that the form 
9f public cena mee of the most significant influences upon this complex process of 
ions s. the view that examinations are ends in 


Curriculum change. This being so, persistence of 
emselves, ee their wider influence, should be challenged. D a 
The growth of integrated science courses during the past decade or so“, for reasons and 
circumstance hich f country to country, has provided an opportunity to depart from 
e gine A egre traditional with the separate sciences. The changes in 


O; 1 
Se examination practices which were 


be. 


al intentio: 
hip or nepotism. Where the trend towards more open 


as in North America, the selection function 
although competitive examinations 
dominate their educational systems, 


1 
S.S. Dunn (ed), Public Examinations, Australia, Rigby Ltd., 1973, p. 206; and W.B. Elley 
and I. p, Livingstone, External Examinations and Internal Assessments, New Zealand, NZCER, 


ges Ch. 3. 
Richmond, New Trends ..., Vole I, OP- ches 
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practice depend, among other factors, on past history, degree of selectivity in the secondary and 
post-secondary systems, the ability of science teachers to cope with broadly-based science courses 
and the competence of the examiners. Clearly, no set of circumstances in one country fit exactly 
those in another, and no prescription for measuring students? progress can be independent of 
circumstances. The huge empirical study of the International Association for the Evaluation of 
Educational Achievement (IEA) on science education in nineteen countries! demonstrates a range of 
practices, Experiments with forms of school-based assessments which may be feasible in some 
countries may be premature in others. But there are some criteria and techniques for 
assessing performance which are widely applicable, irrespective of country and subject area 
(see Ramsey, Chapter 7). - -—" 
This chapter provides a review of some of the influences of public examinations, their efte 
upon teaching practices, points to some related problems, and considers the way ahead. 


The effects of public examinations on teaching and learning 


A recent view that teaching and examining should be considered together ? (e.g. Wrigley), — 
represents a significant shift from a long-standing obsession with the reliability of examinations. 
Because criticism has been focussed on the unreliability of examinations 3, this has been for a 
long time the main target for reform with consequent improvements in examining techniques. lving 
Unfortunately, these developments, desirable as they are, are seen by many teachers as! inve as 
complex mathematical abstractions comprehensible only to the highly numerate. For this vg , 
improvements in examining practices have not often been related to the practical problems © io 
Science teaching. ^ One of the causes for the rapid decline of general science over twenty Lp» 
ago was that the public examinations did not reflect the intended approach to the teaching o am 
courses. Most of the examination questions set required the recall of factual upset T 

the separate fields of biology or chemistry or physics *. The examinations did not mate A 
teaching approach which was intended to have ®its roots in the common experience of UR 
and (sought) to elucidate the general principles observable in nature, without emphasizing tive 
traditional division into specialised subjects ...9?, The social, economic and administra t 
forces for the retention of established ways of examining proved too strong to be ee S 
educational ideologies. ^ Public examinations by written essay-type papers were thought (^ val 
impartial. It was useless to argue, as Eggleston did, that impartiality is not Justices non by 
be equality of injustice  , Power and prestige continued to be accorded to the examinat 


1 York, John MENS 


L.C. Comber and J.P. Keeves, Science Education in Nineteen Countries, New 
1973, Ch. 7. 


2 " T 
J. Wrigley, in Schools Council Examinations Bulletin 32, Assessment and Test in the 


Secondary School, London, Evans/Methuen Educational, 1975, p. 10. 


R.L. Edgeworth, Essays on Professional Education, London, Johnson, 1808; 


E.C. Rhodes and C. Burt, The Marks of Examiners, London, Macmillan, 1936; and 


: H S 
M. Dunstan, Sources of Variation in Examination Marks*, University New South Wale 
Bulletin No. 2, Australia, 1966. 


3 p. Hartog, 


r i i o. 138; 
L. Connell and W.S, James, General Science Today*, The School Science Review, N 


London, S.M.A., 1958, p. 277. 


1936; 
5 urray; 
Science Masters? Association, The Teaching of General Science, Part I, London, = 


p. 30. 


6 . ience 
J.F. Eggleston, A Critical Review of Assessment Procedures in Secondary School Se 


United Kingdom, University of Leicester School of Education, 1965, p. 10. 
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Teachers felt they were more accountable to their students 
ant to hand over to the teachers control of the examinations. 
ce accumulates that public examinations do 


as eviden 
hey are often irrelevant to the needs of students; and 


st-secondary education, still less in non-academic 


universities and the professions. 
than to society which was reluct 
This reluctance is gradually being overcome 
not measure certain important qualities 1t 
they are not good predictors of success in po: 
Occupations. 


How can public examinations be used to improve science teaching and raise the level of 


performance of students? The development of integrated science projects in many countries of 
the world over the past decade or So has been widely publicized. Experience with many of these 
new courses shows how student endeavour can be stimulated and performance raised. So often, 
however, the public examinations set do not measure the new outcomes and abilities gained, and 
the results are not an accurate reflection of the student's real performance during the course. 

In the sections which follow, some suggestions will be made for bringing examining techniques 


into line with the purpose and practice of the new integrated science courses. 


Techni xus 
ques of examining 
integrated science projects generally follows the 


nt and diffusion pattern (R.D. and D.) (one which is particularly 
) the relationships between ways of examining and objectives of 
and teaching methods, are too frequently neglected. Until 
E of ways of examining with precisely-defined course 
examinations will continue to be unsatisfactory 


to measure. 


Although the process of course development in 


familiar research, developme 
Uited to the teaching of science 
he Course, and ways of examini 
importance is realized of the i 
objecti E methods public 
ives and also with teaching jt € corpo 


in = E " 
that they will measure only part um can be given of how to improve public examination 
In this chapter, only a basic OU F information needed is in fact not difficult to understand 
ves that the andling the elementary techniques given the 


histicated treatment of the 
o examples of which will 


Procedur a 
es. The writer belie of bi 
A upon, and that all teachers 0n en itidem. 

ingness to modify their ow? outlooks e widely available, 


techniques, technical works Or testing aT 
e given, rom any examining procedure depends solely 


on The credibility of marks oT . Absolute reliability (defined as consistency’) 
twi sources of error due to variations in marking, 


i © properties - thei reliabili! 
i es eir ad ` t 
5 impossible to achieve because of three bro se abilities which erc a bere of 
conten didates. Some o e ways by which these 
Candia formance by C9" i More detailed method 
at i erfor dealt with later. ore detailed methods 
Sources phus rege” = i ated and oie will be found in specialised works 2. 
of anal of error might be si the degree g En tion marks is validity, that is the degree 
The S vais o to estima’? dibi of examina" mination of 'student progress 
eco; i the , 
th Which ps pa M is meant to do. the abilities required to answer examination 
e est does W! f deciding " 
Proble m omes at O inte ated science © 
m of validity bee tives Be specifically defined in 


Quest. i 
ions 1 the objec 
' are consistent with o objec Lo doe questions which make demands consistent with 


For more sop 
reference t 


resulting f 


Te 
Stricted sampling of syllabus 


xts referred to above to determine 


minat; 
ion, Only when cours 
8 y w T e 
the, Pected to be able to do iS it poe bed in the specialized te 
* Empirical procedures 977 ; à scl 
OW fa pirical p ighede f this chapter is to discuss ways of raising the 
[i RE ee -— or re rest nations in integrated science. 


e 
Teliay he primary purpose, theref f public €% 
ility and validity of the Te? 


1 

Elle " ch. 4. 

Q Y and Livingstone, 0P- elisa qho Certificate of Secondary Education: an 
ulletin 3, E H.M.S.O-, 1964; and E.J, Furst, 


oB. Sch i jnations P Lo 

I chools Council Exam jques» J 
curoduetion to some Examini Too ev york, McKay, 1961 
“custructing Evaluation InS8705 7 


ss in integrated science—public examinations 
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No prescription for measuring students! progress can be independent of national circumstances 
but there are some criteria and techniques which are universally applicable. The matching of 
ways of examining to course objectives, it has been argued, is such a principle. It can be 
demonstrated in generalized form by means of the matrix in Table 12. 


Table 12. The relationship between ways of examining and aims in integrated science courses. 


Ways of examining 


Recall of basic information 


2. Understanding of key concepts 


3. Higher cognitive processes 
(e.g. thinking critically, solving problems) 


Mastery of practical skills 


Ability to design experiments 


6. Scientific attitudes applied to Society 


c examination, 
e the term ‘objectives’ would 
f six ways of examining 
Brief comments 
ng follow together with 
particular skills or 


Clearly, this table of specifications is much simplified. For a particular publi 
it would be desirable to state the aims in more specific form befor 
be justified but Table 12 illustrates the possibility of using a cluster o! 
each of which is suited to measure how far certain aims have been achieved. 
on the advantages, but not the disadvantages, of each method of examini 
examples of test-items designed to measure the degree of competence in 
attitudes. 

cts and understanding of 
wers of inference an 
tion because the 

se can be 


Multiple-choice tests are particularly convenient for testing recall of fa 
concepts although test items can be constructed by experts to measure po 
reasoning. In simple form, they are widely used as part of the examina 
answers can be marked easily, quickly and reliably, and a wide range of the cour 
tested in a short time. 


Example 1 tests recall of facts. 
The stages in the life history of the housefly are in order, 


(3 larva-egg-pupa-adult. 
(b) pupa-larva-egg~adult. 
(c) pupa-egg-larva-adult. 
(d)  egg-larva-adult-pupa. 
(e)  egg-larva-pupa-adult. 


98 


Evaluating student progress in integrated science—public examinations 


Example 2 tests understanding of principle (induction). 


the relative motion of a conductor and a magnetic field, the 


When a current is induced by 
h a magnetic field that opposes the motion. 


direction of the induced current is such as to establis 
This principle is illustrated by, 


(a) a magnet attracting a nail, 

(b) an electric generator or dynamo, 
(c) the motion of a compass needle, 
(d) an electric doorbell. 


(Examples 1 and 2 from Schools Council Examinations Bulletin 4.) 


Essay-type written papers continue to be commonly used, often in conjunction with objective tests. 
The questions set may be in a variety of forms - structured (guided) questions to test different 
levels of ability in the different sub-questions; questions which require comprehension of data 
Provided and the drawing of conclusions from it; and, less frequently, free-response of open-ended 
Questions. Although too many questions in these papers continue to test no more than basic 
information, they are used to best advantage for testing higher levels of cognitive ability and 
qualities such as power of organization and ability to write lucidly. 

structured question). 


Example 3 tests higher cognitive processes ( 
the loss of heat from a sample of water a 


n air gap on 


In order to investigate the effect of a 
gure 5). 


Pupil set up the experiment shown (Fi 


Figure 5 
thermometer 


asbestos 


air gap 


nt of water at 60°C. The temperature of 


Two identi j i amou! 
ti d with the same 
ntical beakers were fille plotted on a graph. 


each was recorded every minute and the results were 


1 
" i t, London, E.U.P., 1969, p. 2 
J.F. Eggleston and J.F. Kerr, studies in Assessment 9, p. 25 
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Figure 6 


Temperature 9c 


(Example 3 from Nuffield Secondary Science, Examining at C.S.E. level, 


100 


70 


65 


60 


55 


50 


45 


40 


35 


30 1 2 3 4 5 6 7 8 9 


Time, minutes 


iti same? 
How does the air gap affect the loss of heat? Is it increased, decreased, or the 
In experiment A, what was the temperature after three minutes? 


Oc; 
How many minutes did it take for the temperature to drop 15 C in beaker A? 


" ight minutes? 
What was the difference in temperature between the water in A and B after eigh 


2 
" aii t loss more 
Which of the following materials, if placed in the air gap, would reduce the hea 


Underline the correct answers. 
Vermiculite Glass wool Copper 
Foamed polystyrene Water Rubber 


: se-builder 
Explain how knowledge of the results of the above experiment would enable a hou 


to reduce heat losses in a house. 


2. 
London, Longman, spit) 
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Example 4 tests comprehension of data and drawing of conclusions. 

a 'territory! in the breeding season. They patrol 
lves and their young chicks. Other owls rarely 
the number of territories on a 

and the average number of young 


Like many other birds, pairs of owls establish 
the territory and from it obtain food for themse 
trespass. The data below shows, over a period of years, 
particular estate, each one of which is held by a pair of owls, 
birds reaching the fledgeling stage per territory. 


Figure 7 
bo 
5 
E: 1.5 
E @ 1949 
e 
o 
£ 
bo 1.0 
CE 
$3 
$t 
“8 
E a 0.5 
82 01951 
o 
o 1955@ 1958 
Eg 
Of 0 24 26 28 30 32 


18 20 22 


Number of territories on the estate 


(a) Describe how the number of chicks per territory reaching the fledgeling stage changes with 
wW e n! 
time. 
ith time. 
(b) Describe how the population of owls changes with f 


" i i leads to 
Suggest how the change in population with tine 


ing stage- 
number of chicks reaching the fledgeling 5128 
ce Project (SCISP) Paper 5, 1972.) 


(Example 4 from Schools Council Integrated Scien 


(c) an explanation of the changes in the 


ul nning i i lected assignments 
i and is most effective when se 

dee E d a the course are specified for inclusion in the 

ivi 


Course 
tudent to keep up with work. It offers 


frog = -work assessment need 
m a variety of different types of act 


as incentive to 2 § x 
ee It provides a strong ve rednooh the peak of examination pressure for teacher 
re and repeated feedback. is not an alternative to tests but a measure of some of the 
lent at the end of the course- It i a systematic record-keeping of profiles of grades 
ired. 


*58 tangible skills and attitudes acqu 


awarded ; 
d is necessary. ry of practical skills which are not 


ree of mas te: 


Pract: the deg: ae h ; 
Shared tests are intended to pp given to individual practical work varies greatly. 
with theoretical work. 


nt nineteen countries completed. the optional practical tests `. 
e I.E.A. study only two out 2 ncs measure abilities quite different from traditional tests 
evidence suggested that practic 


1 


Comber and Keeves, op. Cit., P- 29. 
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and from tests designed to measure practical skills without access to apparatus. (Professor Sim 
used sets of colour transparencies in a study in Malaysia, 1974.) If the unique value of science 
in the curriculum arises from acquiring knowledge through experiment, observation and 
problem-solving, the skills involved should be precisely specified and evaluated 1. A form of 
internal assessment closely related to the practical work carried out on the course is generally 
preferred to external practical examinations, which have a restricting effect on the range of 
laboratory work carried out in schools and usually base the marks awarded on the results Oo 
If an assessment of the practical ability of candidates in public science examinations is not 
attempted, experimental work may be neglected on the course. Five methods of assessing 


practical work are described by Macintosh?. 


nly. 


Example 5 is a multiple-choice test item intended to measure an aspect of practical ability. 


In its gaseous state it is denser than air and soluble 


A substance has a boiling point of 21°C. 
le apparatus by 


in water. From the diagrams below, select the one representing the most suitab 
which to collect a sample of the substance. 


Figure 8 
A B C D E 


Saturated brine 


Ice Cold water 


(Example 5 from Comber and Keeves, op. cit., p. 377.) 


Projects provide an opportunity for study in depth and usually enhance motivation. TRAY Fadl 
scope to show originality in the design of experiments or field work and to demonstrate prac gem 
skills. They can be very time-consuming for both teacher and student but they measure à = 
range of abilities. The topic for the project should be precisely stated, feasible and cover 
limited problem for the practical study of which resources are available. 


Example 6 tests ability to design experiments and practical skill. 


Devise two ways to measure the speed of a model racing car and compare the results. 


s 
versity Pree” 


i " 
E.g. J.F. Kerr, Practical Work in School Science, United Kingdom, Leicester Uni don 
1963, p. 21; and H.G. Macintosh (ed), Techniques and Problems of Assessment, Lon?" 
Arnold, 1974, p. 98. 


2 " 
Macintosh, op. cit., p. 90. 
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Oral tests are unreliable, often not feasible, time-consuming and favour extrovert students. They 
assess verbal fluency, self-confidence and can reveal scientific attitudes and reasoning powers not 
Otherwise apparent. Although the tradition of oral examining continues in some countries 

(e.g. in eastern and western Europe), particularly in relation to practical ability, it is not 


normally recommended by the writer at this level. 
To provide a more specific example of the task of relating an examination set in several parts 


to a detailed specification of objectives, the revised version of the SCISP pattern is reproduced 

below. This! course has been referred to frequently in this volume of New Trends in Integrated 
Science teaching as well as in Volume Il. It is a course of integrated science of double O-level 
value for 13-16 year-old pupils. The standard provides a satisfactory basis for A-level science 


courses in the United Kingdom. 


AIMS 

It is hoped that pupils will achieve the following aims: 

Skills 

eir degree of competence in 


ts which would enable them to pursue science 
cal Sciences) to a higher level or as a 


Pupils should be able to demonstrate th 


1. (a) recalling, and (b) understanding those concepts 
Chemistry of Physi 


(courses in Physics, Biology, 
hobby; 


2. (a) recalling, and (b) un derstanding portance to the scientist; 


those patterns which are of im 

3. making critical appraisal of available information (from whatever source) as an aid to the 
formulation or extraction of patterns; 

itical appraisal of available information (a) in order to solve 

oned judgements; 

order to communicate them to others; 


of science in relation to technical, 


using patterns and making CT 
Problems, and (b) make reas 


5. organizing and formulating ideas in 
$. understanding the significance, including the limitations, 
Social and economic development 


ng of scientific work; 


". being accurate in the reporti eT v—Á 
8. desieni P le experiments (in the laboratory and elsew! ere (a) t solve 
Mos ced ka Mn perseverance in these and other learning activities. 

Attitudes 
Pupils should: 
f a group; 
9. be able to work (a) individually, and (b) as part of a gro 
10. (a) be sceptical about suggested patterns, yet (b) willing to search for and to test for 
Mit: ledg ithin the community 
ll. be " f scientific knowledge within k 
` lication © 
pongeran Toe M EP building blocks, interactions and 


fundamental ideas - 
ai on mres jll be a continuous search for 'patterns' which are 


Thy content of the work is base oid W 
"oe td Throughout the p Lap 11 then be used to solve problems of a practical and a 
iios. nt generalizations an t aime tO educate through science as well as in science, and so 
is b sess Tha a s sociological implications and technical applications of science. 
arly concerne 
* Schools Council Integrated Science project (SCISP), Patterns: Teachers! Handbook, London, 
cil In 


Longman, 1975. 
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Ideally it is hoped that a single teacher will eventually be able to deal with the course. 
There are possibilities for team teaching, but it would, of course, be contrary to the nature of 
integration if the material were divided into subject areas separately taught by specialists. 


WAYS OF EXAMINING 


Integrated Science leads to a double credit at O-level. A single credit is available. At present, 
five papers are set, together with teacher assessment. 


Paper 1 Essay/short answer paper testing aims 4b, 5, 6. 

Paper 2 Essay/short answer paper testing aims 4a and b, 5, 6, 10a. 
Paper 3 Multiple-choice paper testing aims 1a and b, 2a and b, 3. 
Paper 4 Multiple-choice paper testing aims 1b, 2b, 4a. 

Paper 5 Essay/short answer paper testing aims 4a and b, 5, 8a. 


Teacher assessment testing aims 5, 7, 8a and b, 9a and b, 10b and ll. 


One of the commonest criticisms of public examinations is their undesirable backwash effect 


on methods of teaching. It is important that the techniques of examining should be compatible 
with the teaching methods used to achieve particular purposes. For example, there is little 
possibility of the spread of investigatory methods of teaching which lead students to find out for 
themselves if the examination demands mainly recall of facts. The possibility of matching ways 
of examining to teaching styles can be illustrated by means of a matrix. 


Table 13. The relationship between ways of examining and teaching methods in integrated 


Science courses. 


Ways of examining 


Teaching methods 


1. Instruction/teacher-directed, didactic 
2. Practical demonstration/teacher-direct, 
participation 


3. Practical work/teacher-directed, finding out 


4. Project work/student-directed, finding out 


5. Discussion/collaboration, theoretical 


6. Solving problems/teacher-directed , 
theoretical 


he Private study/any of above | 
4 


fferent abilities» 


Sings different teaching styles are likely to result in the development of di ; desirable: 
examination pattern which encourages a teacher to use a variety of teaching methods iS tyles 
luate teaching 9 


As far as science teaching is concerned, one of the studies carried out to eva 
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ttainment and attitude tests was completed by Galton, Eggleston 
-six teachers and 3,000 students. Although the findings 
dence that even with new science courses which 


emphasize guided-discovery methods, many teachers do not question students about their 
Observations during practical work or encourage them to hypothesize or design experiments. 
The conclusions suggest a lack of match between the curriculum developers! aims and practice 
in the laboratory. The need for a close inter-relationship between the three components 
7 examinations, aims and teaching methods - should never be overlooked. 

An international view of examination patterns in integrated science, including all types of 


Combined science courses, reveals a rapidly changing scene and a growing awareness of the 
de in this chapter within a wide variety of 


Practices, A great deal of the integrated science examining is at Junior Certificate level 

(as in Kenya) after only two or three years science. A common pattern is that which is found 

un the United Kingdom, east and west Africa, the Caribbean countries, Malaysia, Hong Kong, 

Singapore, Australia aud New Zealand. It comprises two or more structured papers, usually 

including some objective questions, together with a form of practical assessment. A trend in 
by internal assessment. Although 


many of these countries is to replace external examinations © e 1 
there is evidence of the wider use of different types of questions, little attempt is being made 


to assess higher, cognitive skills, for example by testing the Sb i handle data oin Aay 
conclusions (as ‘a Examples 3 and 4 above). A number of countries, including the United Kingdom 
India, the United States, France and Sweden are using internal assessment as a component of the 
total examination era * runded on fC comparison with (or moderating) the externally-awarded 
Brade, On the other hand, in most central European countries and elsewhere where grading is 

by internal assosSHiEE only the process is being widely criticised because of its unreliability, 


variations i d influence on pupil/teacher relations. The pooling of 
S in sta tween schools an i a ; 
national Sheep Africa and east Africa as well as in the Caribbean to form regional 

examination boards has resulted in ant to local circumstances. 


the setting of questions more relev: 
i he schools in another group of countries, including the United States, Canada and Japan, use an ` 
y 1 
nternal system of credit awards which ar 


e often checked by externally-prepared standardized 
rn Europe, China and Cuba is towards the 

asus. A trend in the socialist co 
€ssment of groups of students, & P 


autres EE said to lead to the development of 
"ei assessment and a sense of semi. are mentioned which need to be taken into 
Skene on ste renee are compared and improvements in public examinations 
m integrated science are considered. 


in relation to student scores on a 
and Jones! with the co-operation of ninety 
are written in tentative terms, there is evi 


ence 
een about increasing the validity of public examinations. 


als are mostly about reliability and arise from 


the examining process. 
expertise of science teachers as examiners. This 


P r 
Toblems of examining integrated 50} 


So " 

far in this chapter our concern has b 

ii problems of implementing the propos 
Banizational and technical limitations 'U 


" of t 
is Peta most obvious difficulty is the n which has à higher measure of internal assessment 
m Ore serious in a decentralize syst h usually works through an examinations board. The 
: m whic dealt with later. Another organizational 


n in a closed centralized syst@ 


i : ing methods is 
™Portance of training teachers in ning 


exami 


processes and Products of Science Teaching, 
M. Jones: = (M 1976. 


1 
M.J 

:J. Galton, J.F. Eggleston and i millan), 
United Kingdom, Aan Council Research Series (Mac 


2 7 ‘ x 
" Techniques and Criteria Used in Assessing Student 


Se 

ku E. Lister's paper on vena Pio International Congress on the Improvement. of Biology 

levement in Biology Education Y information about examinations in the countries mentioned 
r fur! er 


ducati 
bove ion (UNESCO), 1975, fo 
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problem is ensuring that adequate information is readily available to students as well as to teachers 
about the examining system, particularly about the nature and purpose of each part of the 
examination. It often leads to unnecessary apprehension and tension if students are not aware 
of the skills which each part of the examination is intended to measure. It is desirable that more 
than one examiner should mark scripts independently (except for objective tests) and candidates 
should not sit more than one examination per day. This is not usually possible because of cost 
and timetabling difficulties. 

A number of common fallacies arise from a lack of appreciation of the 'numbers game' pes 
in public examinations. The total mark obtained by pooling scores in a number of tests each o! , 
which measures different abilities (for example, by adding marks from course work, a project an 
a practical test) is almost meaningless. A profile of grades is more informative. Errors in 
the reliability of marks are caused by differences in the standard of marking between mamia 
and in the spread of marks over the mark scale. The continued prestige in some quarters Q 
the percentage mark - 40 per cent is a pass, 70 per cent a distinction - is a strange bit of - 
fiction. The application of simple statistical methods can help to minimize many of these uy 

No more than a brief reference is possible to other standard methods of improving reliabi + 
which are a little more technical, some of which involve the application of statistical techniques. " 
One of the major problems with school-based examinations is to achieve comparability yel 
marks from different schools. Relative achievement levels between schools must be reliably 
known if the grades are to be acceptable for the award of certificates or selection purposes. ils 
Methods of moderation are available to determine the standards of achievement reached by pUP. 
in a particular school relative to those of other schools2. In particular circumstances to 
(for example, in the different regions of Malawi where educational opportunities available dd 
students vary from one region to another), each region is allocated a different achievemen 
for comparability purposes. 

Another problem area which influences the validity as well as the reliabi neal 
is the complex matter of sampling. The sampling of skills and teaching methods has a uc t 
been mentioned. The sampling of course content is considered by Ramsey (Chapter 7). Tot 
a choice of questions is offered on examination papers, it is equivalent to setting à Se at 
quite different papers, often a very large number of them. If a candidate is asked pa papers: 
five out of ten questions on two papers, this is equivalent to offering over 60,000 ques pn 
It is also important to ensure that a test-item, particularly with objective tests, is not 


diffieult or too ea ili it discriminates between good and poor i 
o easy (the facility value, F) and that it discrimi iota Ji doscribed simply i? 


lity of examinations 


candidates (the discrimination value, D). This technique of item ana integrated 
Examinations Bulletin 33. The problem of measuring the affective outcomes of an zy em are not 
Science course has not been mentioned. "Attitudes to science" scales are available Uu ts and 
generally used in public examinations. The assumption is that gains in scientific P9 
attitudes as a result of the course will be reflected in the candidate's performance in the 
examinations. 


The way ahead jence 
s in scienc" 
There can be no definitive answer to the problem of how to measure student progres f economie 


‘i o 
or any other subject. A system found acceptable in one country because of its stage 


| ; i d Macintosh, OP- cit.» 
See, for example, Schools Council Examinations Bulletin 3, op. cit., an 


Ch. 8 and 12. 
2 " = 
These methods are described fully in Elley and Livingstone, op. cit., Ch. 8-15. 
3 
Schools Council Examinations Bulletin 3, op. cit., p. 51. 


4 
N.F.E.R., 1973. 
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development and educational history may be unsuitable for another country. While public 
examinations in the United Kingdom at present are becoming more open, giving more responsibility 
to teachers, some European countries are discarding course assessment because the skill of 
teachers to control quality and reliability is in doubt. Since the methods used reflect the values 
fter a substantial lag in time - there is a 


and assumptions of existing society - though usually a 
need to keep examinations continually under review. It should be the responsibility of a named 
tions on the basis of community needs; 


group to initiate discussion about the reform of examina 
for example, by a body such as the Schools Council for the Curriculum and Examinations established 


in England in 1964. This body has pioneered the reform of examinations in integrated science 
through SCISP to encourage a much wider range of outcomes and it is now financing the after-care 
of the project to facilitate diffusion of the new methods. The Caribbean integrated science project 


(CARISO) sponsored by Unesco fulfils a similar role. i EN m 
Looking ahead, it seems likely that the guidance function of public examinations will increase 

as emphasis on the selection function decreases. The real problem is to develop in science 
] competence and perceptions necessary to cope 


teachers and examiners the understanding, technical compe : a 
With and indeed encourage this change. If we don't, judging from past experience, examining 


Processes will be relatively unaffected. There is an urgent need for all teachers to be familiar 


With the basic principles and techniques of examining mentioned in this chapter. An introduction 
1 A es of assessment should be included in initial training 


to examinati niue 
minatio: the principl a E 
Courses for per cogn wi forming 4 regular feature of the further professional training 
of Practising teachers Lecture courses on these matters are inadequate. Structured workshop 
Sessions are " d in which small groups work closely together on real problems, at the 
preferred ln m cises and discussions. It is necessary 


Same ti i A :nciples from practical exer 
for Mei drawing kapep abge workshops to assemble a collection of materials which are relevant 
e 


to local ci baon One of the sections of the Activities 
End Experienoss collec aan pero Feedback to teacher and puj il (FTP) includes units on ways of 
examining, a te etse Questions i sted in the examination, devising a good 

> eacher's oral q i marking an 


Question "sin u^ 
, designing an examination pape 
LAM units, sample parts of which are reproduced below 
asis for a workshop. 


Sample 1 Sample 2 -— mv 
Students " 11 the What skills are being testec in each o e 
lie. are asked to review 4 Gai following items? Categorize eack according to 
each s the exhibition, make 2 D the highest ability which you think it tests. 

ype of item, what they think it (22 items in: ünib-) 


te me 
Sts and where they might use it in 


their teaching. 
A red powder is known to be an oxide of 


Senn zeli. It dissolves in dilute nitric acid giving 
This could be a walk-round exhibition, a pale blue solution. Which of the following 

Pa alternatively items could be passed could it be? 

Sed the group. The following OP A. Ferric oxide Fe,0,- 
rel cd ond -— oxide 0090: 

is Cc. Mercuric oxide HgO. 


Essay questions. 


2. 
Structured questions. p. Red lead Pb40,. 
3. " 
Fixed-response questions. 
1 9 in Science Teacher Education project, op. cit. 


Feedback to Teacher and Pupil. topic 
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4. Questions requiring pupils to produce 
drawings or diagrams. 


Short question tests. 
A practical examination paper. 


Various homework assignments. 


oa Oo a 


Personal and impersonal accounts of 
experiments done in class. 


9. Accounts of projects. 


10. A short tape-recording of a 
question-and-answer session in 
class. 


11. Question books, e.g. those of the 
Nuffield O-level Physics Project. 


12. Comprehension questions about the 
content of a passage. 


Through the Eyes of the Pupil 
contains material suitable for several of 
these categories, as also does chapter 9 
of The Art of the Science Teacher. 
Tutors using the unit in trials included 
other material on assessment in the 
exhibition, e.g. booklets from examination 
boards, complete GCE and CSE papers 
and published collections of questions so 
that students know where to get material. 
The exhibition would be broadened by 
adding intelligence tests, creativity tests, 
attitude tests, ete. 


Discussion, and briefing for private study 


A closing plenary session provides 
opportunity for the students to ask 
questions about the materials in the 
exhibition. They can then be asked to 
devise either a short question test or a 
longer written test on a topic of their 
choice, applying what they have learned 
in this session. 
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E. Zinc oxide ZnO. 

2. The construction of the evolutionary family 

tree for reptiles and mammals is based upon 

A. Degrees of similarity and differences 
between the structure of their bodies. 
Evidence from blood tests. 

C. The evidence of fossils only. 

D. The knowledge that in evolution small species 
are replaced by larger species. 

E. Experiments with inheritance. 

3. Man's food resources would decrease if 

A. Green plants became more numerous. 
Herbivorous animals eaten by man used 
energy more efficiently than at present. 

C. The food chains between plants and man 
became shorter. 

D. The numbers of steps in the food chains 
between plants and man increased. 

E. Light penetrated deeper into the ocean 
than at present. 

4. When the speed of a car is doubled its 

kinetic energy 

A. Remains the same. 

B. Is V2 times as great. 

C. Is twice as great. 

D. Is four times as great. , 

i acceleration” 
E. Is increased in proportion to the 
on, 
5. In order to launch a rocket to the mo 


it is proposed to carry out the 1 
satellite in space. 


aunch from 2 m 
The main advantage of 


method is that 


A. 
B. 


The distance to the moon is less. E 
on 1 
The gravitational attraction of the mo 


greater. ocket- 


d in firing the 7 


rocket is le59: 


Less accuracy is neede 
The thrust needed for the 


Loading operations are simpler. 
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Sample 3 


Guidelines for the criticism and improvement 
of test items. 


5 Structured questions, built from short 
nswer items, to each of which the candidate 
constructs his own response. 


1. Does the stem describe unambiguously 
and economically a situation that is 
intelligible and interesting to the 
candidates ? 


Are the items following it short, 
unambiguous and answerable in à 
Sentence or two? 


3. 
Do they test worthwhile skills? 


4. 
Is there a gradient of difficulty from 
the early items to the later ones? 


B. 1 
Fixed-response items in general 


(but mor : t E 
items), e particularly, multiple-choice 


1. : 
Is the bulk of the information in the 
Stem, so that the options are short? 
2, 
Do the options follow fluently and 
grammatically as extensions of the 
Stem? 
3. 
Are the options plausible? 
4. 5 
Does the item discourage guessing, 
and encourage the type of thought 
" it is hoped to test? 
` Can the correct option be identified 
in any other way besides reasoning 
as intended? 
6. . , 
Does the item test a worthwhile skill? 
T. 


can a candidate 


In a series of items, 
evertheless 


Who gets early items wrong " 
Still score on later ones? 


attention in 


Other ing for 
matters calling seus 


Part 
ticularly to control intermittent 88 


e immediate 
ment of € 


Sample 4 

Marking by impression and with a mark scheme. 
(Copies of about five scripts supplied to each 
student.) 


First reading 


Read through the scripts quickly, and award 
each a mark out of 20 by 'impression', i.e. by 
a general reading forming an overall judgement. 
Do not award marks for specific points at this 
stage. Enter the marks on your mark list. 


Sorting method 


Now try a different method. Look at each 
script and decide whether it is average, below 
average, or above average, and place it on one 
of three piles accordingly. Then take the pile 
of average scripts and spread these scripts in 
front of you in rank order. Treat the top and 
bottom piles similarly. Examine carefully the 
borderlines between the piles, and make 
adjustments as necessary. Award the middle 
script 10 marks, and the other scripts 0-20 

as you think appropriate. The extremes of 
the mark range need not be used. Enter the 
ks and position in class on your mark sheet. 


mar 
ur ratings with those of your 


Now compare yo 
colleagues. 
Criteria 

ke a list of the criteria you used in 


Now ma 
s to some papers than 


awarding higher mark: 
others. 

ng a mark scheme 

Compile a mark scheme for the question, using 
a total of 20 marks. The marks allocated for 
each point should be specified. 


Devisi 


Using it 

Mark the script that comes first in 
alphabetical order, using your scheme, and 
then compare your scheme and the mark 


awarded with those of your colleagues. 


Finally, you may like to agree on a mark 
scheme with your colleagues and then remark 
the whole patch. Compare the marks with 
those awarded by impression. Is it worth 
having 2 detailed scheme? 


future are the question of record-keeping , 


ourse and practical work; the monitoring of 
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question banking1, that is the collection of a library 


laboratory work and its relation to theory; m 
nt 


of test items of known statistical characteristics for use by examination boards and teachers; 
more understanding of the non-cognitive aspects of learning science. Examining the broad spectrum 
of student progress in science learning could be so much more than measuring what is known. 
Research and development work on examinations has already provided useful guidance, some of 
which is referred to in the following bibliography. Owing to growing public demand for 
accountability in education, the conflict of role between the testing and guidance functions of 
examinations seems likely to continue. It is to be hoped that reform will be guided more by 
co-ordination between examination practices and educational enquiry than by political expediency. 
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SUMMARY 


This chapter begins with a discussion of : 
differences between pupils. What this means in 
Tepresents the adaptation of the learning environme 


Which evaluati al part. A series O 
features of cpm denen which are throught most relevant to catering for differences 


between individuals Various programmes, at primary, middle and secondary level, are outlined 
and described by profiles using these dimensions. Finally, some of the ways which teachers can 


» isti ils which affect learning are 
use i i :ndividual characteristics of pupi ng 
to gather information about the mec programmes developed in different countries. 


discussed and illustrated with examp 


the importance of adapting learning experiences to 
practice is described in terms of a model which 
nt as à succession of adjustments, a process in 
f dimensions is defined describing variation in 


INTRODUCTION 
le for evaluation to play. Among the many 


Where T ; there is a TO SER : i 

different decisions are e npe of integrated science face daily in their work is an 

importa ds of decisions wie learning experiences of individual pupils. Within any class or 
nt set which concerns the te t, level of development of concepts 


des i ;n such things as interes > ' 
and cogniti E ae a soe E attitudes, previous experience, general intelligence and rate 
ive and manipulativ E 
y in 


to which they allow for variation 
of 1 F : e manner and degree 
earning. Course materials VAT d alternative a 


between =i flexibility activities built mo kiris oig teachers 

may cler. 2 BET oan i activities to suit the ee siam vas Mnt LA 

Some there is prone aa Provision for accommodating y* fuit pupils involves taking decisions : 
ny attempt to d ide learning experiences viu ve d using of information about individual 
ased on aroraa about the pupils. The gathering pee 


of this chapter. 
Pupils in this context is the kind of evaluation which is the concern p 


h d here differs from that discussed in other chapters in that 
The role of evaluation considere integrated science teaching, built into the strategy in 


it i i or T F 
eee of a teaching-learning amer dh T uplls with experiences appropriate to various aspects 
of tare win of the hoe ebur T 1 i - fect their learning. It is for this reason 
tha: levelopment an A ; the rationale for ' p ; à ; 
die begin with a brief discussion Of del of the way it is carried out in practice, showing the 
i rences, followed by a suggeste this, various approaches to the teaching of integrated science 
El Which evaluation plays- After f indivi dual differences will be considered, taking examples 
bos methods which take account = o final section, we draw together various methods which 
ie eed different countries. m as part of a strategy for helping each pupil along a unique 
used for evaluating indivi 
Path of development. 
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The importance of individual differences 


One of the strongest and most obvious reasons for the use of teaching methods which take variations 
between individuals into account is that such variations are important in determining achievement. 
Variations which arise from differences in home background, measured intelligence and other 

aptitudes , have a far greater influence upon learning 1 than teaching methods and school environments. 
In a later chapter of this book, Brown shows how very little of the variation in attitudes to science 
of thirteen-year-old pupils was accounted for by the characteristics of the science course or class, 
whilst about half was ascribed to characteristics of the pupils which were present when the pupils 
entered school. Brown's results are typical of others? Which have shown that differences such as 
those already mentioned and others which could be added - 

through various modes, personality factors su 


The latter pupils may be equally 
(e.g. a longer time for working, more 
ty to encounter the same concepts 


concrete manipulation, larger or 
in a different context). 
characteristics which add 
reflected in performance. 
individual characteristics to a greater degree, mors pupils 
learning, each in the manner which suits, and starting fro 
Another reason emerges from the changing emphasis on different goals of education; science 
teaching, it is urged, must be concerned, in addition to cognitive 
responsibility , self-discipline, decision 
goals, "processes by which new proble: 
past9", The goals are more likely to be achieved throi 
take responsibility for their learning, 
than through conventional methods. 
integrated science teaching. 
differ radically, 


lis. Bloom, 


Summarized by H.J. Walberg, 
in K. Marjoribanks (ed) Enviro 


Stability and Change in Human Characteristics, New York, Wiley, 1964. 


Es - Kagan, "Reflection - 


impulsivity; 
Journal of Abnormal Ps 


cholo; » Vol. 7", 


^y.p. Andrew, "Schools, Science 


and Societyn 
P. 319-324; and L.E. Klo; 


» Science Eg i 
pfer, "Science Education | cation, Vol. 
1969, p. 319-324, 


: 94, No. 4, 1970, 
cation in 19917 


; School Review, vol. 77, wo, 3/4, 
5v.M. Howes, Individualization of Instruction 


New York, Macmillan, 1970, p. 5. 
112 


The role of evaluation in adapting learning experiences in integrated science 
to differences between pupils 


Adapting activities to pupils in practice 
It is important to be clear about what is meant by adapting learning experiences to individuals as 
well as to say what is not meant. What it is taken to mean here is that the range of experiences 
for which the individual learns is such that demands are made at the right level to encourage 
progress. Since each pupil is unique, it is unlikely that identical sets of experiences are equally 
Profitable to all, and thus there will be differences, sometimes large, sometimes only slight, 
between the activities of pupils. What this individual catering does not mean, is that the pupils 
will be working alone. Partly because catering for individuals has sometimes been implemented 
through programmed instruction and teaching machines, there is a common tendency to associate 
‘individualized! learning with tjgolated' learning. As defined and interpreted here, adapting to 
individuals is not related to the separation of students during the process of their learning nor 
does it depend on the use of educational technology. In fact, no assumptions are made about 
class organization beyond the essential provision of flexibility and opportunity for the teacher to 
have individual contact with each pupil; this could be within the context of large or small group 
working, and does not preclude occasional whole class discussions nor private study. An example 
is probably most helpful in explaining the meaning. 

A class of twelve-year-olds was working with cogwheels, looking at the effect on speed and 
direction rotation of fixing cogs of various sizes together, sometimes touching and sometimes 


Connected by a chain. One group had set up an assembly like this: 


Figure 9 


many times D rotated for every one rotation of C. As they 
the cogs one of the group, John, said A was not necessary any 


to find how 


we i at B to turn 
exe ME eed P M d saying "That one makes all the others go round". The teacher suggested 
x ol ffect of taking the chain away and look carefully at how each cog was 


The teacher asked them 


te the € 
Meanwhile John, who seemed quite clear about the cause and effect 


working out how to mark how far D rotated for one turn of C. The 
quickly saw that it was not necessary to move the cogs to find the 
ting the teeth on C and D. It was clear to the teacher that 
t in this situation was at a level such that he would not have 
m ion of the COS; it would probably have been a meaningless 
d in the group had been able to grasp the relationship involved 
t benefited from his approach. 


Robert should investiga 
connected to the others. 

relationship involved, WaS 
third member in the group 
answer and she was busy coun 
Robert's notion of cause and e 
benefited from measuring the rot?* 
exercise for him. But also the E 


have no 
more clearly than John, and ghe ios activities, there are many factors to take into account. 
When decisions are made about p. the level of difficulty, complexity and elaboration of 
ing tO nt when considering the objectives which 


x most obvious are the ones rela to be taken into accou ; i 
possible activities. These have are various ways in which any particular idea can be: 

Ae Pub might achieve, Then, Bere, scuro, NA! ero CI be presented nd which © 
Aen is Beco pa a variety im conta rlenos of the pupils - But o e not all; there are 
ier Eis supe dapervision, encouragement aad incentives: aad "tiene li 


a upils. 

80 7 ronment of the p! y 

me m the factors in the eee pe it comes to making decisions as to how to adapt various 
ere are problems, however; 


It has to be acknowledged that there is a 
aspects of the learning environment 


to the pupils. S ptis 
Serious lack of t evidence from research to answer questions about how prom 
ack of consistent evi 
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learning in pupils with different characteristics. Walberg 1 
has attempted to relate treatment to aptitude for learning, and concludes that very little that is 
conclusive has come out of the considerable effort so far exp 


ended in this area. It seems 
unrealistic to expect that a set of guidelines will emerge for adapting activities to any particular 
combination of abilities and learning characteristics. 
In the absence of prescriptions for the nost 
circumstances, a practical approach is to adapt thi 


through an iterative process. The notion of 'matching' experiences to Pupils is seen as one of 


making successive adjustments to the ever-changing situation. The process can be visualized as 
the repeated cycling through a feedback loop. In this loop, 


reviews the results of research which 


effective course of action in particular 
e learning environment to individual pupils 


As the process of attempti: 
the pupil is some way fur 


after each cycle), 


- 
x 
^ 
^ 
`~ 


^. 
j gathers information 
‘ 
Å about characteristics 

ʻi of pupils \ E 

(d 1 
Pupil interacts \ I 
with learning 
environment 

\\ 

Uu 


/ 


2 / 
adapts features of 
learning environment 
using information 
7 about pupils 


- 


Learning environment 


Since the adaptation of the 
have as many elements of 


the complex process of 


1 
Walberg, op. cit. 
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The role of evaluation 

uation plays a central role. It provides the 

e and skills which is essential for making 

An essential feature of this continuing evaluation 
dpoint or a summary of the effect of actions taken. 
seem rather small and trivial: for instance, 
rk it out independently; to give a working sheet 
investigation; it is the sum total of 
Teachers have, 


tures of the learning environment; what 


| aber, which has just been described, eval 

decisio on about a pupil's present ideas, knowledg 

ipeo aa eine to Daly maka mE next steps. 
at it is a starting point for action, not an en 


a nd e in the former the decisions are 
ave beg pupils as individuals. By contrast, i ee der i 
assi ormation about individuals (at least as far as their perfor ) from marking 
oe setting tests, question and answer 

geneous group and one set of decisions app 


sessions, 
lies to all. 
rated science teaching 


ategy which adapts experiences to pupils 
hes have been taken in attempting to 


A E 
PProaches to catering for individual differences in integ 


As 
are ps Practical problems of implementing ? ere scd 
Sry great, it i i ind that different app 
Surmount hos, it ium ap nde have individual characteristics and adapt methods to their 
Own abilities and eis tity The constraints caused by external pressures and classroom 
thavitions (e nml of the pupils and total years of attendance at school) are among 
t pe Meer aah more suitable than others in particular circumstances. We 


Sha ing experiences to individuals according to the 
try to distinguish between app. to adapting exp 


es of the 1 z ded as fixed or varied for each pupil. i 

earni ironme : i i i 

ha, There are aa a of the teaching-learnt poesia ccm of E oo 

(t Produced a comprehensive list of dimensions - ; r a UT Xa £ 

teaching 5 : prehen: tcm differences, he described each o ese within four levels 
Ocus!, To cater for indivi are under the direct influence of a teacher's decisions 


of fre, 
e 
and it : aeg not be eee discussion to key features of the aem environment 
vith am eros best fo cone ie Prgms and are siblet ral e wie eration In 
Given that pupil parning PY cn, oper i8 m oe, devel of devel ahs 
Chi] Men t pupils vary (see ^in attitudes, interests, level of development o concepts 
and co 8 responses to the same problem f learning, previous knowledge and experience, general 
intelligere and manipulative skills, hei s elements of the learning environment which might 
(at Ma ce and rate of learning), what are timum opportunity to learn? A list of 
St in theory) be adjusted 50 that int ied - in some but not all 


oe which might be affected by the tea 
. i wing: 
nces - would include the follo E same for all the class, or vary between groups or 
e the 


Cire 
(a) : 
Time allowed for learning - C? b 


Vary between individuals. can vary between groups Or between individuals 
Cnet ates mal, opi ae pair, rn Pe 10 fe Sams 
Íor the whole diti, the learning environment - selection of activities, 
une participation in decisions about the Mer holly p teacher, carried out by 
teacher ^ e oer or left largely for fo pur 


Vol. 1, no. 2, 1971, p. 28-49. 
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(d) 


Variations in expectations and goals - whether the content of a 
the pupils are expected to achieve from them can be the sa 
or individuals. 


ctivities varies or not, what 
me for all, or different for groups 


(e) Class organization - can be such that the unit is the whole class, or groups or individuals. 
R " 
f Influence of diagnostic feedback - feedback about pupils can be an important basis for 
> decisions about pupils’ experiences or there can be no feedback gathered or used. 
" ; : n à ils 
)  Teacher-pupil relationship - there can be a marked hierarchy in this relationship, with. eh 
b expected to show respect and acquiescence to teachers, or there can be a non-hierarchica 
relationship, based on mutual respect. 
(h)  Pupil-pupil relationships - mutual help 


be encouraged or discouraged. 


The possibilities which have be 
indicate the range of variation along 
very many different values along each dimension. 

direction from left to right is one of increasing pro 


Table 14. 


Provisions for individual differences. 


en given'here for each of th 
a dimension. 


in the achievement of individual or shared goals can 


ese eight aspects are intended to 
There is no dichotomy but the possibility of 
This is made clear in Table 14 where the 
vision for variation between pupils. 


Variable Range of variation 
(a) Time allowed 


fixed at same amount 


extra time available entirely flexible , 
for learning for all pupils for slower to catch up according to needs of 
the individual 

(b) Content of same activities basic common core for varies according to 

activities carried out by all all with remedial or individuals 

pupils enrichment for some 

(c) Pupils! Pupils take no Part in pupils and teacher 

participation 


(d) 


in decisions 


Variation in 


choosing what they do 


expectations are 


jointly make decisions 
by agreement 


pupils free to choose 
what they do and how 


: minimum level same expectations vary 
expectations same for all for all, more expected between individuals 
and goals of some 

(e) Class s decisions relate to unit for decisions as decisions about pupils 
organization class as a whole a group (3 to 5) as individuals (within 

groups) 

(f) ae of little feedback can modify but not important basis for 
re gathered or used determine course of determining course of 

action action 

(g) Teacher-pupil 


(h) 


relationship 


Pupil-pupil 
relationship 


hierarchical, pupils 
expected to acquiesce 


pupils discouraged 
from discussion with 
each other 


teacher respects 
pupils! opinions but 
makes decisions 


free to discuss but 
no expectation of 


non-hierarchical, 
mutual respect as of 
equals 


pupils encouraged to 


discuss and help each 
mutual help other 


The list might well be extended, but serves to Show the variety of elements which affect 
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In some circumstances, a teacher's freedom of 


learning but over which a teacher has control. 
the school, but generally these 


action may be restricted by constraints set within or from outside 
Operate to reduce the range of variation which is possible, rather than entirely remove freedom. 
In most situations, teachers have to make some decisions about the range of items which have been 
listed, If there is in practice no variation between the learning experiences of individual pupils, 
it may well be that this is because such variation is thought either to be unimportant, impracticable 
or undesirable. 
These variables can be used to describe the characteristics or different integrated science 
Programmes and to reveal the extent to which a programme allows for variations between pupils. 
Two examples serve to show how programmes can be described in terms of positions along these 


dimensions. They are somewhat contrasting programmes for primary schools: 


"The T.P.P.S. course is one of pupil activity. Each weekly 45 minute science lesson ... 
is described for teachers on a card ..- The card gives the title of the lesson, the 
class organization (usually groups of four), a list of materials required, brief instructions 
for the activity, a picture of children engaged in the activity and sometimes questions, 
further information, recording procedures and further additional activities". 

ng, and should be, responsible for 


" (Scie: teachers were capable of bei 
‘ uos 5/13) held that 2 To maintain teachers' freedom and 


thinking out and guiding the activities of their pupils. 
give ce. help Lm ered important, SO that activities could follow and form the 


interests, and match the intellectual development of the children. — the units do not 
constitute a course; they are seen as sources of ideas and guidance which teachers can 


ji i it x 
use in planning and carrying out programmes of work which they devise to suit the unique 


requirements of their own classes". 

all the data nee 
tion has been use 
n Table 15 show the degre 
of the programmes: 


ded for describing the programmes in terms 
d in drawing up the following profiles of the 


P - 
hese extracts are not enough to give 
e to which the variation between 


Io $ 

53 the dimensions, so further informati 

Ed Programmes. The shaded areas ! 
Pils is catered for by various features 


Table 15. — profiles of two contrasting programmes. 


TPPS Science 5/13 
Variable New Guinea United Kingdom 


Time allowed 


Content and activities 

Pupils! participation in decisions 
Expectations and goals 

Class organization 

Influence of feedback ` 


Te 
Acher-pupil relationship 


Py 


Pil-pupil relationship 
: i Research Report 24 
i i . Phase Three Evaluation, Rese p 4 
d Wilson, Three Phase Prima Science : 
2 1974, P- ^ 


niversity of Papua, New Guinea, 


sb 
"n 


H 
arlen, Op. cit., p. 8. 
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i ils with experiences 
i ble teachers to provide pupi Pom intontteiis 
i its were intended to enal r ot a major inteni 
E spes voe and development, whilst for the TPPS this was h has variations in 
ny oE sie = tance difference, but also show that any co imei Kena usas 
The profiles show ; able adaptation to individual pupils. ] ion 
ich i ous features en: s to cater 
the degree to c med Mea integrated science programmes which have been «oe dee 
Euer ecd : HE heivaan pupils, looking first at programmes for children up 
certain differe i 
thirteen. 


Primary and middle school programmes 


i rammes which have been de 2 
al aieo oe E wem s programme! and the Personalized Programme ic cad d 
i -S. (Individuali. es BODA à imo 
eroin are m pe these is based on the philosophy of individualization which imi E ier 
el i ion) 3 i lete K-8 programme arr: 
ivi bed Instruction) ?. It is a comp Female 
in IPI (Individually Prescri : Bid Io uL end em. er 
veral 'units', some to be studie y EAR 
levels. For each level there are se : À [ ride Facio e ir 
, 5 ioi into 'lessons', the first of which includes p 2 oklet! , 
The units are in turn divided in i ts Bon Planning Bo 
i i ly, and use the results to plan, with the he p ae 
student will mark independently , à 3 D. des queque Hc 
i lessons of the unit should be studied. * For eac cone: pt, an ONES 
ue of learning resources - e.g. individual lessons, directed group activity, R^ 
ice a Bone - and the student decides which to Use; after discussion with the teacher, u^ 
pes There will be certain Science content lessons whic IET 
nt of concept attainment is built into these lessons. 


veloped for individualized 


to say that, because of - 

l students may follow unique paths en 

what the path is to be. However, student choice of diet 1P enta: 
is not created from the raw ingre 
adapted to the characteristics of " 
Beyond this, the student has to adapt 


gramme could be use 


d to 
The aim is to Provide pupils with experiences matche 


bin. Klopfer , "Individualized Science; 


relevance for the 1970s", Science Education, Vol. E: 
No. 4, 1971, p. 441-448: and A.B hampagne and L.E. Klopfer, Idiviqualized Learning 1i 
Individualized Science. Learning Research and Development Centre, University of Pitts 
U.S.A. (Unpublished). 


2 M.C. Linn, B. Chen 


and H.D. Thier, 
Presented at meeting 


"Personalization in Science: 
of National Associ 


a pilot study". Paper . r4, 
ation for Research in Science 


ilt 
Teaching, Chicago, APT 
31.0. Bolvin and R. Glaser, "Development aspects oj 
Audiovisual Instruction 


f Individually Prescribed Instruction" , 
> 1968, Vol. 19, p. 828-831. 
T Áistruetion 
Champagne and Klopfer, Op. cit, 


118 


The role of evaluation i i 
on in adapting learning € i in i 
H x i 
to differences between pupils E ae eae Scots 


their in| 
um Vade lunae and the approach chosen to implement this is to "allow children to chi 
in this escapes a general framework of possible choices. One assumes that ed 
the progienime ras will choose activities which will help them to learn" 1, In the development a 
Other things, to R found, however, that pupils not used to devising their activities tended, a * 

, rry out only superficial investigations with no follow-up on preliminary mu 


It was thi 
erefore decided to provide the pupils with guidance as to what to investigate and how to 
The guidance takes the form of written 


Star i i 

e a given a particular set of equipment. 

on the is Spi a problem and giving suggestions for starting on it; further hints are given 

though the the pupil gets stuck. The activities are designed to be carried out by one pupil 

Write a r y can be performed in groups. Students are encouraged to complete a challenge and ý 
eport of their work before selecting their next one. The only restriction in their choice 


is that 
no more than one of each set of equipment is used at a time in the class. 
shows, on-going evaluation is effectively absent. 


Nn iy summary profile of this programme 
Certain rules the pupil who makes decisions about the next actions, except in so far as there are 
S and obligations, for example, a report has to be written after each challenge. It is 
after all, no-one knows better 


argued 
how stig at the pupil is in the ideal position to make the decisions; 
the pupil feels to tackle a new problem, what is really interesting, etc. This is true 


but th 
the on decisions should also be taken with the goals of learning in mind, and in the knowledge of 
ntial opportunities offered by different courses of action for achieving these goals. It is 


doubt, 
real 2 whether pupils have the conspectus necessary to make optimum choices and there is a 
Sk that progress will not be made if the decisions are left entirely to the pupils. Therefore 


again: 

ie a advantages of pupils having first-hand access to knowledge about themselves, one must 
them. e disadvantage of their ignorance about the over-all goals and various routes to achieving 
marked contrast to the programmes so 

o not play a large part; the teacher 
ific content to be taught to children; 
The project materials comprise a 
units to use, when to use them and in 
llowed to work freely with materials 
nd in introducing new activities 


ect (APSP) i$ in 
s by the teacher di 
ot just a list of scient 


tay an African Primary Science Proj 
as a idered. In the latter, decision 
5 "APSP is D 

working with children". 
Teachers choose which 
hat pupils be à 


t what is found a 
ns. He can help them make 


ut th 
e teacher's role in developing thi " 
ils test their explanatio! 
ties which the children have not thought 
of the problem they are trying to 


about but 
È which may h them gain 4 É 
decig, It is han s ue p He c has to evaluate the situation and to 
e e when to make interventions - the programme Wi 3 _ Unfortunately , neither does 
ac; b gramme give any guidelines in maki these important decisions; the process by which 
Vities can be liie " individ d the decisions are left to intuition. 


Simpl 
e 
apparatus. He ca 
better unde 


ent that the 


eing 

ui 

quite unstructured. Considerab ih ronce” ar 
ific expe 


inter 

ests 

Start; and stages of development; spec aa i 
"ih ting points os o de E pem their own activities. In other words, pupil choice is 

oi e pupi hoice not from a prepared range of activities but from all the 


ura; 
Bed, but in this case it is € 


earn to interpret experiments?' (mimeo) 


1 
do children H 
OF Berkeley, California. 


M 
` Linn, + 
; "Indivi izi i : j 
ividualizing Science ance Hall of Science, 


avail 
2 able from the author at Lawr 
E.A y ; " 
" ; Evaluation Re ort, University of Ib: 
Yoloye, Evaluation for Innovation. APSP ty adan, 


Nigeria, 1971 
Afri ide. 7 d Papers' 
can pr; i " rs' Guide: Inks and Papers’ as reproduced from th 
Primary Science Project, ao vol. II, OP- cit. x 


tria] eg: 
edition in Richmond, New Trends -- 
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activities thought of which are at all practicable. The same doubts apply here as expressed f 
earlier concerning whether pupils can effectively not only match but also stretch themselves in this 
free choice situation. Moreover, the Nuffield project emphasizes that "Once the children are 
interested in and handling things it is imperative for the teacher to discuss with them" 1. The 
guidance during discussion has, of course, to be appropriate to the individual child; but, as in 
the case of APSP, no help is given to the teacher with this 

Science 5/13 also depends on teachers to make decisions about appropriate objectives and 


activities for their pupils. In this project, it is intended that goals, ways of working, time for 
learning and interactions of all kinds should be set b 


y reference to the pupils' characteristics. 
The project provided help in this matter by givi 
development (using Piaget's framework) and by su 
different stages. Evidence showed that for 
for pupils at different points in development 
a project whose concern is specifically 
environments suited to individuals 2. 


Table 16. Summary of features of pro, 
(Please refer to Table 14 for the interp: 


rocess of providing learning 


grammes for pupils up to age 13. 
retation of points along each dimension.) 


Variable Individualized Personalized African Nuffield Science 


Science Science Primary Junior 5A3  . 
Time allowed ] 


Content of activities 

Pupils! participation in decisions 
Expectations and goals 

Class organization 

Influence of feedback 
Teacher-pupil relationship 


Pupil-pupil relationship 


MNN 
CURRENT 


4 ý 
Nuffield Junior Science Project, Tea: 


chers Guide ns London, Collins, 1967. 
2 
Progress in Learning Science, Check- 


lists for Earlier Development, Draft edition, 1976. 
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are the Australian Science Education Project (ASEP), a programmed adaptation (PHI) of the Scottish 
Integrated Science Project, and the Korean Mastery Learning Project in Integrated Science. The 
features of these programmes are summarized in Table 17 in terms of the same descriptions as 
used for the primary and middle school programmes. A comparison of the profiles in Table 17 
with those in Table 16 shows that individual variations are accommodated to a smaller degree in 


all of the secondary projects than in the ones for earlier years. 


Table 17. Summary of features of secondary programmes. 
(Please refer to Table 14 for the interpretation of points along each dimension.) 


aan NU __ LLL TIED RERSCTUSM 
: ASEP PHI Mastery Learning 
Variable Australia (Scotland) (Korea) 


Time allowed 


Content of activities 


Pupils! participation in decisions 


= a 

| ma m =) 
Expectations and goals Em] a 

| — - 

gH  J 


Class organization [E 


Influence of feedback 


Teacher-pupil relationship 


Pupil-pupil relationship 


The i roject is à science programme for 13 to 16 year-olds designed to 
allow for were among students. The forty-one rn. sa popile j iis second ee 
materials and handbook, do not constitute a course, but are separa seci al nu A es ing 
about 10-15 hours of lags time; they can be combined in various ways ha c E oe e up 
his or her own course "A typical unit begins with a section (known as e core) which is 

` bout one-quarter of the time a student spends 


compl is planned to occupy ® : 
on eger p dee TEM varies with individuals, as students are expected to attain a 


Minimum level of competence The remainder of the time is spent on options, which form most 
c . 


op’ e un Students are inv i interest them. Some 
3 rest of th i invited to choose the options that in! ! 
tions a : = ded t tend the ini we ble while others provide less intellectually 
re intended to exten 


tellectually capa! 
demanding or more concrete activities" "- TEOS 
each “hone 86: diagnosia = = i poe the end of a section, generally only at the end of 
ch st sts C x 5 
the co — However, Wiss rce the instructional material", but do not supply continuous 
re; they serve to "reinio of the work. It is acknowledged 


feedba, ja to adapt activities during the course 3 
hy noe bep i. gua pencil-and-paper type tests do not evaluate achievement of 


i using other methods, which are 
Don-cognitive objectives and teachers are urged to do this by using 


Mainly to be found by reference to other sources. 


hich are self-administered and marked by 


R 4, p. 10. 
Australian Science Education Project, A Guide to ASEP, 1974, p. 1 
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The Scottish Project pH (Programmed materials for the Highlands and Islands), aimed to 
adapt sections of the Scottish Integrated Science Scheme (see Jeffrey, chapter 12) and to produce 
supplementary aids which would enrich the learning environments of pupils in isolated and small 


secondary schools. The project explored the value of programmed materials in compensating for 
inadequate laboratory provision and understaffing. The programmed materials produced had the 
following features: 


"1. They impose a measure of structure on the learning situation. 
2. They require active participation by pupils. 
3. They incorporate feedback on the pupils! 


expected responses or sensations" 2 
The pupils work individually through a work book which gives step-by-step instructions and 
information. At intervals, the pupils take tests based on the content which should just have been 
learned; they mark the tests themselves. . At these points, or at any other time when they 
encounter difficulties, they ask their teachers for further explanation. 
introduced in the programme, for example, 


to teach the use of certain equipment for those not 
already familiar with it. The PHI materia 


all pupils and the modes of learning, though varied in 
different parts of the programme, cannot b interaction between pupils iS 
determined by the programme. -going evaluation of the grasp of facts 
her and to the pupils, thus helping them 
to take responsibility for their learning. 


developed to meet the opposite 
Project? had perforce to ada 


‘or 13 to 16 year-olds has been devised which 


1 
M. Roebuck, J, pi 


loomer and D. i i ; 
of Education, 1974. Hamilton, Project PHI, University of Glasgow, Department 


» Technical Supplement 17, Department of 


9H. Kim, Experi ng Y ng in Korea 
h " erimentation in Educati ing i 
Seminar on "The Evaluation of the CENTRES rre presos d at EP, Pare, 
September/october 1974. m 


Education" held at HEP, Paris, 
4 
J.B. Carroll, "A model of School learning" 


Teachers College Record, vol. 64, 1963, p. 723-3 
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Figure 11: Flow chart of mastery learning strategies. 


1. Diagnosis of 
learning deficiency 


2. Compensatory 
programmes 


3. Presentation of 
instructional 
objectives 


4. Teacher instruction 


5. Formative tests 
8. Co-operative 
learning 
9. Summative tests 


Most of the components are sel 


7. Enrichment 
programmes 


6. Remedial 
programmes 


"presentation of instructional 


tory, though the 
f-explanatory Using the teachers! guide, the 


quire enlargement. 


bjecti á ing! may Te 
teacher ye and 'co-operative veio egit each section of work, what they have to do and the 
ells the students, before they oals are the same for all the students, since it is a 
The £ ve mastery if they are allowed 


Standa: 
r 
ds of performance expected. t "mos 


ast 
Sic premise of the mastery strategy tha DS Thus the ability to achieve 
e 


an : 

sius Spend the necessary amount of tim au variable in this strategy. A feature which is 

ain objectives is not considered 4 relev S Is the this erative learning component, designed 
mmon g with guidance from the teacher. 


ine], : 
"ded to help students achieve the CO l-group discussion 
pm dualized since the content and kinds 


0 a š 
llow students to help each other in e as being indivi 
Kim2 does not describe this programm put it is the intention of the mastery learning 
tivities are not varied for di ferent pups: as is practicable and this is 
vironmen ensatory enrichment and 


tegy io adi 1 en to the a Er ond 
a t the learning K e use 

9n P ^ arning an " 

e © through the variation in time for le anja 11; the result does not compare unfavourably with 


P media] programmes. As iS s ' ' ? 
which claims to be individualized. chool levels differ in many eatures, as Tables 16 
Th tlined for all $ à ing learning opportunities which are 
ad 17 S the program n pr "importance of a pupil having 
Show, they agree upon e 


l Mastery Learning . New York, Holt, Rinehart 
s, J.H. Block (ed) Mas 


B i 
ana ya om, " Mastery Learning : 
inston, 1971. 
71 e School, final report on Mastery Learning Project, 


2 
H. ki ing in the Middl h in the Behavioural Sciences, 1970. 
M m, et al, Master, Learn Institute for Researe 


Arch-August 1970, Seoule, Kore 


of ac 
Stra 


een in 


a 
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adapted to present capabilities as much as possible. In order to provide a pupil with such 
opportunities, it is necessary to know about him or her as a learner - the how, what and wherefore 


of learning. Various methods are available for finding out about the pupils characteristics, and 
it is to the discussion of these that we now turn. 


Methods for on-going evaluation of individual pupils 


As a starting point for discussing the pros and cons of various methods for 


gathering information 
about pupils, we can consider what, ideally, is required of any method. 


a It should be capable of providing information about 
learning, which will include pupils! interests, 
and cognitive and manipulative skills, 


the key characteristics relevant to 
attitudes, level of development of concept 


style of learning, previous learning, rate of learning, 
motivation for learning. 
2. It should enable information about relevant items from those mentioned to be gathered 
frequently and repeatedly. 


23. It should give detailed information; 


anything as general as an IQ Score, for instance, blurs 
out variations existing in one person between abilities of different kinds. 


The possible methods of assess 
Structured, through the semi-st: 


ment can be envisaged as Spread along a dimension from the highly 
points in this continuum are: 


ructured to the unstructured. Typical characteristics at three 


highly structured: 


time and administration rigidly controlled, generally group tests, 
marked against externally set standard (examples: external exams, 
IQ and other standard tests). 


Semi-structured: 


; can be group or individual, marked 
iven criteria (examples:  teacher-made 


unstructured; carried out at any time, 


c generally individual, assesses performance 
in normal activities against behavioural criteria (examples; 
observation schedules and check-lists, informal interviews). 


U Glaser and R.C, Cox, 
outcomes", R.A. Weisgerb, 
Rand McNally, 1969. 
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From the list of ideal requirements for a method of evaluation, it is clear that highly 
Structured tests are quite unsuitable for the purpose being considered here. Semi-structured test 
are more suitable; these may be either built into the programme and taken at specified point: pe 
provided by the programme and used as desired by the teacher. The I.S. programme Pu ane 
Korean Mastery Learning Project all provide examples of tests built into the wee "ha 
test item shown below comes from the PHI unit ‘Electricity in Motion' and is placed just) after ni 
part of the programme where the effects of number of bulbs and numbers cells connected have 1 


been investigated separately and in combination. 


Figure 12 
Block F; Test Card 6 ] 
1. Here are some circuit diagrams. Look at each one, and arrange them 


in order according to how much current would flow. Just write the 


letters A to H inclusive in your jotter, starting with the one with least 
current, finishing with the largest current. If you think any two 
circuits will give the same current, underline the two circuits. 


A B 
PUE 
D E 


Project PHI, Department of Education, 


Source; PHI, Workbook 'Electricity in Motion' , 


Uni i 
niversity of Glasgow, 1973. 


ded for tea 


chers to use as they choose, the 'Evaluation 


8 As an example of tests provi 4 
"pplements' of the elementary science projec 


t SCIS (United States) can be taken. 


The method 
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i i her 
sted for checking progress could be used equally well for on-going evaluation. The teac : 
sugge tructed to prepare a specified set of the evaluation materials and ask certain questions an m 
is io» criteria for judging satisfactory understanding of the points involved. This example com 

from the evaluation supplement for the unit 'Organisms'. 


"Invite one or two children at a time to a quiet corner and give them time to examine the 
materials on the tray. Tell them you are interested in what they have learned about 
planting seeds and growing plants. First ask: 


1. Name the materials you would need if you were going to plant these grass seeds. 
Do you need anything else? 


After each child has had an opportunity to identify the Objects he would use, ask: 


2. How would you use these materials? or 


Describe how you would plant the 
seeds and grow plants. 


Now ask the following convergent questions: 


3. How deep would you plant the seeds? 


4. Here are three bags of soil that have different amounts of water in them. Which 
one shows best how wet the soil should be for growing plants? 

5. How long after planting grass seeds will it be before plants begin to grow out of 
the soil? 

Criteria. The criteria below will help you recall each child's responses: 

L: Did he select all of the necessary materials for planting: seeds, soil, planter, water? 

2. Did he describe the process of planting? 

3. Did he recognize that the seed should be covered with soil but should be near: the 
surface? 

4, Did he select the damp soil sample? 

5. 


r the grass to emerge that approximated the 
classroom evidence?" 1 


What are the advantages and disadvan 
evaluation? Practical co: 


built into the programme 


these tests are able to probe children 
detailed way. 


tage which these tests share with all eee 
; furthermore, since the tests have to be Sho" 


UD Mor » concentration and performance. In 
» the method is sui » concepts and knowledge, but not for the other 


learning. For these it is necessary to turn to the methods which are 
the continuum outlined earlier. 


1 
SCIS Organisms Evaluation Supplement 1972. 
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TE It pan be argued that special situations do not always have to be set up for finding information 
a N progress, for if the normal activities really do provide opportunity for the goals of 
no e to be achieved, they also provide opportunities to see how far development has reached 
os rds these goals. In circumstances where it is practicable for observations to be made during 
SE UNE learning activities, this can be the basis for a method of gathering information which has 
Pa pics advantages. Observation does not interfere with the pupil's work and can be carried out 
erent occasions, giving a variety of behaviour samples. A range of many different 
characteristics can be assessed by observation, and it has most of the features suggested as 
desirable for the purpose under discussion. On the other hand, observations are subjective and 
less reliable than other forms of assessment. 'Observation' in its fullest meaning includes 
listening and responding as well as looking, and is a skill which requires training. While it may 
Dot.take up the teacher's time when it becomes a normal part of work, some training or mental 


Preparation may well be required before this state is reached. 


There is a great danger that observation provides little more than personal opinions, with 
Therefore, if it is to be the basis of a viable 


low reliability and, consequently, low validity also. 
method for gathering feedback about pupils, it cannot remain totally unstructured. It requires a 
Structure which comes from considering the goals of children's learning and from developing 
Se which can be applied in interpreting children's behaviours in terms of progress towards 
e goals. 
is Duckworth provided teacher 
le the affective goals of the programm 
Ssons to find out whether they were 
"Here are some questions 4 teacher can ask himself a 
day to day: 
1. Does he make suggestions about thi 
somebody else what he 


ver a problem and kee| 


s using APSP with general guidelines, which are clearly related 
e, to help them use their observations of children during 


making progress: 
s he watches a child's work from 


ngs to do and how to do them? 


has done so they can understand him? 


2. Can he show 
p trying to find an answer even when 


3. Does he puzzle 0 
it is difficult? 
4. Does he have his own ide? 
for help? 
5 D h M s not agree with something that has been said? 
= joes he give his 
6. Is he willing to change his mind ab 


s about what to do, so he doesn't keep asking you 


in view of new evidence? 


— —— n RM time? 
23. Does he ever watch something patiently for a long 


24. Does he ever say, "That's beautiful" ? " 
^t a more detailed level, the project «progress in Learn prece ned F t 
developed check-li oe : porti a framework for observation. Eac P sip quein ih PA 
nty-five abilities : ts and attitudes which are among = ae a hm cum 1, 'identifyi 
Activities che ilities, concep d 13 years of age- Some examples e ngo 7 ying 
pupils between 5 an unicating - verbally and non-verbally', 'interpreting findings! , 


Vari; 
ablest! traigi H ' 1 1 ' iosity' 
B. Gon S', 'raising questions', d tenergy' s 'adaptation' , and attitudes of curiosity’, 
' cepts of 'area', 'volume', force » rate! Statements in the check-list are 
9Penmind, ? | «willingness to C0-0P8 P © | i 
ledness', ‘responsibility’, at three levels in the development of each item. 


tended to indicate typical behaviours 


1 
E. Du kw Primary Science Programme, EDC, 1970. 
ckworth, Evaluation of the African Prim: 
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For example: 


Interpreting findings: 


3 Rarely goes beyond a straight report of observations showing little Sign of appreciating 
any patterns within them which could have been discerned. 

2. Makes rather sweeping statements about his observations, Suggesting patterns which are 
not justified by the evidence. 

3. He puts 'two and two together' and attempts to find patterns in his findings which are 
justified by what he has observed. 

Openmindedness: 

A Tends to stick to preconceived ideas ignoring contrary evidence and behaving as if 
unaware of the existence of opinions or findings different from his own. 

2. Changes from one idea or opinion to another inconsistently being influenced by the authority 
behind them rather than the force of the evidence or argument. 

3. 


Generally listens to and considers all points of view; accepts ideas different from his 
own if the evidence is convincing. 1 


The check-lists are intended to structure observation, focusi 
to be significant in relation to the goals of science 
constantly in this way as a basis for guiding teache 
being made of the observations. 


ngly recognized as bei 
as pupil self-image, their reactions 


their status within a group, 


is kind of i ion i i 
procedures, but there are examples from other fiel oatottatfon t their 
into account. The programme 'Man; 
Strategies' 2 for teachers. 


Five different Strategies 
understanding, opinions and 


Interview - a way of giving pupils chance to expr 
Classroom Environment Check-lists - for pupils 


Creative Formats - to explore Pupils' imaginati i 
Content questionnaire - to diagnose difficulties iil hat ie 
Classroom observing - providing informati 


1 progress in Learning Science, Op. cit. 


2 
Man; A Course of Study, Evaluation Strategies 
Centre, 1970. ~~ Seles, Cambridge, 


Mass, Educational Development 
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to differences between pupils 


reactions to the learning situation and in adapting their approach in consequence is the Ford 
Teaching Project 1. This project provides a wealth of ideas and evidence relating to the 
identification and removal of constraints which impede inquiry/discovery learning. The application 
in science education of these approaches to evaluation and the further development of new ones may 
well help in devising future integrated science programmes which are able to provide all students 
with opportunities for intellectual and personal advancement. 


Conclusion 


Approaches to teaching which are sensitive to differences between individuals aim to provide the 
conditions which are most conducive to successful learning in all pupils. It has been argued that, 
when these conditions prevail, then not only are cognitive goals achieved more readily, but the 
experience of success and the atmosphere of mutual help promote the development of positive 
attitudes; indeed these approaches provide extremely well for the 'curiosity, compassion and 
competence', which Albert Baez selected as key words for future integrated science courses. 

The essential feature of catering for individuals as interpreted here is that the learning 
experiences are adapted to the characteristics of the pupils; in approaches where this is not 
attempted, the pupils have to try to adapt to the learning experiences, and for many this is not. 
Possible. Since it is not only the activities but the surrounding organization and the whole setting 
of learning which affects the pupil, it is necessary to speak of the "learning environment' as the 
tar i individual characteristics. 

ae ee A eae few definite guidelines as to the adaptation of various features 
of the environment to match different aptitudes and modes of learning. Nevertheless, the | 
matching can be carried out in practice by on-going evaluation, the constant gathering of information 
about pupils and use of this in making decisions about the opportunities and guidance of individual 
Pupils. Thus evaluation has a vital role in prosit eng edd basis for action and is at 
thi i rs for variations between in à 

) i pies cepere circumstances mean that the approach which seems 'ideal' 
in one country or one setting may be impracticable in another; therefore approaches have to be 
devised to operate within these constraints. Some of the child-centred Roue mns operate 
in certain schools in western countries, depending heavily on decisions sem y bs € and 
Perceptive teachers, are not feasible in regions where class sizes eu oe eachers have 
Minimum training and resources for a wide choice of actives ere mite j vc Ju 

Methods of evaluation which can be used are similarly cons aie ay i. z TN 
However it has been shown that, for the purposes of providing immedia x T $ eae : 
methods of testing are inappropriate; more suitable D da gy Pm Fac a ani dim gs 
Use informally, frequently and without subtracting und L x ena a eee Fei dp a hed 
evaluation is essential to approaches sensitive to indivic ae we oris ea 
Suited to achieving the objectives which will be increasing'y hes size A rcu be E 
Matter of urgency to devise or to adapt methods of evaluation to Serve p soa hte 


1 
Ford Teaching Project, Unit 1 
Unit 4 Teacher Case Studies, 


tterns of Teaching, Unit 2 Research Methods, Unit 3 Hypotheses, 
Es for Applied Research in Education, University of East Anglia, 


1975, 

2 $5 NT - Teaching as part of General Education - Looking Ahead, in 
"*. Baez, Integra [o it. 

Richmond, New Trends ... VOl- i, op. © 
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nonetheless a good Starting point. Gives outlines of three approaches; 

(Bloom), Individually Prescribed Instruct 
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10 Evaluation of the African Primary 
Science Programme (APSP) 


Ayotunde Yoloye 
University of Ibadan, Nigeria 


SUMMARY 


This is a description of efforts in curriculum programme 
School science programme (APSP) which operated in over 
Programme was as much an experimental programme as th 


evaluation with respect to a primary 
ten African countries. The evaluation 
e science programme itself. 


INTRODUCTION 
The African Primary Science Programme (APSP) was one of the three major curriculum 
development projects in Africa sponsored by Educational Services Incorporated (ESI)~, Newton, 
Massachusetts, in the sixties. The other two were the African Mathematics Programme and the 
African Social Studi amme. 

oe K t, APSP had a number of unique characteristics. 


As a curriculum development projec 
involving over ten Anglophone African countries. 


ers were an international group, and trials of the 
al African countries simultaneously. 


l. It was a Pan-African programme 
Therefore, the curriculum develop 


materials were carried out in sever 
a truly experimental! approach in the development of 


2.. " adopted 
2t curriculum a nete S itia up only after the ideas had been experimentally tried 
eir materials. DB Such trials took place at international workshops or 


with children in actual classrooms. > ate 
in science centres in various participating co 3 


i f materials. There 
3. i jecti ere formulated before the production o! 
D qol eol of preparing "good" materials for the African classroom. 
y the ) 
4, é :ficati als was eventually formulated with the help of the 
A more detailed npacificanon x s programme had been in operation and evaluators had 


evaluati ears after amm e 
had He nn ri reer the operation in the trial classrooms. 

5. The focus of APSP is the child rather be 
Science was visualized simply a9 ĉ too. 
desirable lines. 


n the discipline of science or the society. 
r helping the child develop along certain 


Overview of evaluation procedure 

urriculum de 
ational workshop, 
kshops € 


velopment programme was expressed right from the 
one member of the team was included to consider 
xplored evaluation further and more specialists 


poncern for evaluation of the C 
9Binning. At the first intern 
Problems of evaluation. Subsequent wor 


t 
Later Education Development Center (EDC) 
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were brought in The evaluation team that eventually emerged was international as well as 

ught in. y ge A : 
inter-disciplinary in nature. Contributors came from the United Kingdom, the United States, 
inter 


D education, ye. , Sociology, 
ada a various ai ountri They w drawn from education ps ology, S 

African c es he ere t 

Can nd f hi 

economics and science. 


The composition of the evaluation team fluctuated over the years, but the headquarters 
remained at the University of Ibadan under the direction of the author. 


The evaluation procedure was unique in a number of ways. From the beginning, n — 
evaluators realized that because of the emphasis of the curriculum programme on the eiie E —— 
of the child rather than the concepts of science, conventional tests of cognitive ise pect hune 
be largely inappropriate. There was need to develop novel evaluation instruments and stra eit 
to cope with the innovative feature of the programme. The evaluation procedure therefore an 
experimental in much the same way as the curriculum development procedure was experiment d. - 
A large variety of instruments and strategies of evaluation were tried out, modified or — 
tried again, and so on. Perhaps one of the most Significant outcomes of the evaluation exerci 
is in fact the philosophy of evaluation that eventually emerged. 

Initially, it was believed that the purpose of evaluation was to convince African governments 
of the worth or otherwise of the programme so that the programme could be adopted or rejected. 
Within the first two years of the programme's operation, however, it quickly became clear that the 
essential factors in the adoption or rejection of the programme had little to do with evaluation. 

The main factors were: 


(i) the involvement or otherwise of the policy makers from the ministries 
in the programme; (ii) the absence/presence or 


quality of competing science programmes initiated 
or supported by the ministries in the countries; and (iii) diplomacy or the skill of human 
interaction. 

As a result of the above, it was felt that the major role of evaluation Should in fact be to 
inform the curriculum developers themselves of the value of what they were producing and help 
guide the production so that it could move in the direction of perfection. 

This was a significant switch fron consumer oriented evaluation to 


producer oriented 
evaluation. A consequence of this switch was that the evaluators there 

formative evaluation. 

zormauve evaluation. 


closely involved in the process of curriculum development. They could not remain simply > g 
"objective outsiders", While the assumption of this role to some extent may affect the objectivity 
of the evaluator, it has the advantage of being less threateni 

therefore less 1 


handicaps. 


to a variety of definitions depending on who is defining it and for 
undertaken. 


as indicated earlier, the definition of 
T the purpose of APSP evaluation. He says, "Evaluation is the 
process of ascertaining the decisions to be made Selecting related i 
analysing information in order 


1 Marvin C. Alkin, "Product for 


improving educational evaluation" i t 
September 1970, Vol. 2, No. 3, p. 1. valuation" , Evaluation Comment, 
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Table 18. 


Evaluation of the African Primary Science Programme (APSP) 


APSP Framework for Curriculum Evaluation. 


Observation 


Ointment and teclas) 
nstruments and techniques 


ANTECEDENTS 
(relevant conditions 
Prior to the 
introduction of 

the programme) 


TRANSACTION 
(Various kinds of 
interactions and 
Activities involved 
in the development 
and presentation 
9f the programme) 


OUTCOMES 

(the effect or 
Introducing the 
Programme on 
various categories 
of People, 
Organizations 
Policies, ete.) 


1. 


5. 


6. Economic implica 


. Learning 


Manpower 2 
(teaching, 
supervisory 
developer etc.) 


. Children - 


environment 


Instructional ~- 
materials 


Instructional 
techniques 


Supportive 
strategies 


Intervention 
strategies 


Cognitive 
changes in 


Affective 
changes in 


Psychomotor 
changes in 


Effect on e 
policy 


ducational adminis! 


Quality, quantity, 
qualifications, 
predispositions 


Intellectual ability, 
socio-economic 
background, cognitive 
styles 

School setting, 
educational administration 
economic resources, 
political and cultural 
setting 


Nature, quality, cost 

and development strategies 
Teacher-child-material 
interaction 

Teacher training, 


interaction with policy 
makers, interaction with 


resource people 


Sequence of introducing 
materials into schools 


Child, teacher, parents, 
curriculum developers , 
administrators 

Child, teacher, parents, 
curriculum developers, 
administrators 

Child, teacher, parents, 
curriculum developers, 
administrators 

of children, 
earning capacity, 


tration and 


tions 


1. Questionnaire surveys, 
direct observation, tests 


2. Questionnaire surveys, 
tests 


3. Direct observations, 
questionnaire surveys 


1. Readability indices, task 
analysis, cost effective 
analysis, expert judgement 

2. Direct observation using 
observation instrument 
e.g. interaction analysis 

3. Direct observation. 


4, Survey 


1, 2, 3 and 4. 
Test, direct observation, 
designed experimental 
studies, surveys 


5. Surveys, direct 
observation 


6. Surveys, designed 
experimental studies, 
cost-effective analysis 


KENNEN" Sa 
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A framework for evaluation based on a model proposed by Stake was drawn up to define the 
full scope of evaluation and identify procedures for collecting evaluation data. ] 

Table 18 shows the framework, with a list of various evaluation instruments and techniques 
employed in data collection. In many cases, these had to be designed specifically to fit the APSP 
situation. 


The task of evaluation, based on our definition, was threefold: 


i5 Helping to identify and describe in appropriate language, the curriculum developer's 
assumptions with respect to antecedents and intents with respect to transactions and outcomes 

2. Designing and using a variety of techniques and instruments to collect data with respect to 
the validity of these assumptions and the realization of the intents, The task of evaluation 
in this latter respect, however, went beyond merely checking whether intents were realized 
or not, to identifying unintended outcomes. 

3. 


Analysing the data collected and supplying summa: 


ry information to the curriculum developers, 
organizers, etc., to help in the process of decis 


ion-making. | 
Identification of goals 


It is common to expect that curriculum develo 
describing their assumptions and intents, so providing a clear list of objectives which the evaluator 
‘can use. Experience shows, however, that the curriculum developer is often not equipped to do 
this. The evaluator therefore has to work in close co-operation with the curriculum developer in 
order to evolve a usable description. 


pers should themselves carry out the task of 


adopted an indirect technique of getting the 2 
On the basis of direct observation and analysis 
of the programme's materials and activities (the pro, 


These goals were in two parts. 
nd with intended transactions. 
ended transactions 


APSP aims at developing the following characteristics in children; 
1. first-hand familiari. 


ty with a variety of biological i in the 
world around them; : gical, physical and man-made phenomena in t 


1 
Stake, "The countenance .,.,", OP. Cit., p. 53g 
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Part II 


In order to foster the above development in children, APSP science should have the following 


characteristics: 


the focus of study should be on concrete phenomena themselves; 


the materials selected should capture and hold the attention and interest of children; 


the materials should reveal that there is not always one right answer; 


tunities for a variety of different ways to find out; 


the classroom experience should lead to social interaction among children; 


Te 
2 
3 
4. materials should allow oppor 
5 
4 e simple and familiar; 


to a large extent the materials should b 
T. the materials should encourage children to do things on their own, in their own ways. 


ent in defining its objectives. Did this worry the evaluators? 


d for emphasis implies attention to such powerful psychological 
"readiness" and "learning climate". It consequently 

m where the child's participation in deciding what is to be 
a catalogue of what will be ultimately learnt (especially 


n is as detailed as APSP ever W 
€ demand The choice of the chil 
im woe as "individual differences", 
Piel a predominantly "open" classroo 
fü P» is high, and in which, therefore, 
e cognitive domain) is impossible. 


E : 
Valuation of APSP philosophy 


a the theoretical and philosophical orientatio 

is orientation is a valid one? 

dias ee examined this que 
pirical data that validate the orie 
As with any curriculum emphasizing the chi 


n of APSP, are there any objective ways of telling 


stion and provided for the curriculum developers a summary 
ntation as well as grounds for doubting the validity. 
]d factor, empirical evidence of validity is to be 


found i vides some of this evidence. The fact that the 
majority premi $ E. e i ia will be at the concrete operational stage validates APSP's 
lniistencs that the um of study should be on concrete materials and phenomena. The finding 
at passage from one stage of intellectual development to a higher one may be hastened by 
Appropriate expexienoad with à variety of materials, validates the APSP practice of encouraging 
Tee play with a ety of materials like water and sand, in the earliest classes. The finding 
that à ditiate b ae y ain matura phenomena is usually very different from that of adults 
(and alters x pun o ER the APSP insistence that children be allowed to do things in their 
Own way, Th pay f Maslow ard and others on the concep For how to learn 
validates mess worl x ae zm uld present a variety of problem situations, with the 
Possibility sid S insis enc of ways to find odis The work of Anderson, Lewin 
variety of answers APSP's insistence that teachers be 


aterials sho 


et 
e and Flanders, on classroom Avo 
en integrative rather p Ai were identifi ed. First, Beeby's! hypothesis of 
ee grounds for doubting V9 is for doubt. He hypothesized four stages of growth 


educ x f i "m 
"pee stages typifies One kind eee (ii) formalism, (iii) transition, and (iv) meaning. 
ach chool system, namely, (i) dam. stics of teacher and curriculum. Characteristics 
of ee is characterized gu -school to well-trained in the meaning 
bk beh es ge cran vary from unorganized, relatively meaningless symbols 

: urriculum charac 


by certain 


countries, Cambridge, Mass., Harvard 


1 : 
" elopi: 
hia Beeby, The quality of education iB dev 


Diversity Press, 1966. 
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i i tivity 
i hasis on meaning, understanding, ac 
iri ization in the dame-school, to the emp à x ade 2 e ES 
iem nd oed in the meaning school. APSP curriculum is striving in — 
= Pr eee The basis for doubt is whether this goal is realistic given the pre 
stage (iv) - mi . 


i i i tage (i 
tion in Africa where the majority of teachers in the primary school are in fact at stage (i) 
situation i j 
or (ii) - dame-school and formalism. 


Next, many African countries' policy-makers in education in fact emphasize society in c. 
aot t pem curriculum. The school is seen as an agent of economic development an 
LMCMA EI I3 Š Consequently, educational policies are guided by political and economic f 
Wi n d The basis for doubt is whether a child-oriented curriculum like APSP will be 
ebd id as relevant to a society-oriented educational philosophy. 


A third basis for doubt directly challenges the psychological validity which was earlier 
established for APSP. 


This is a query as to whether psychological findings based mainly on i 
research in western societies are valid for African countries and whether socio-cultural factors 
African countries would not confound the validity of such findings. 

Implications of evaluation of philosophy 


The last two points need to have research carried out into them. 
of studies by African and other scholars on the cognitive developme: 
Otaala, Okonji, Ohuche, Duruji, Jahoda) which show that basically 
in African children. 


It is however the first 


There already exists a number 
nt of African children (Etuk, 
the findings of Piaget hold true 


point that has provoked ma 


jor action on the part of APSP (now SEPA). 
ng APSP had two altern: 


atives open. One was to modify the 
the characteristics of teachers. The other was to change the . 
o fit the new orientation. APSP chose the latter alternative and since 
1971 its major thrust through SEPA has been in teacher 
has been produced, a teache 


education. A trial handbook for teachers 
r's resource book is in the making and, perhaps most significant, 
a centre for experimenting in 


the education of teachers for primary science teaching has been 


established in University College, Njala. Trainees are drawn from all over Africa. 


Evaluation of antecedents 
It was not necessary to conduct fresh studies into 
combination of three approaches was used. 
curriculum development, 


in actual classrooms, 
set-up, 


every one of the antecedent variables. A 
» intuitive evaluation; 


Ors and their assessments , 
+ Second, existing publications wee 
s annual statistics of education, annual reports of education, developmen 
i itical policy statements, were studied 
Other existing publications were 
S Surveys of Socio-economic backer: 


15.3. Foster, Education and S 


ocial Change in Ghana, London 
and E.A. Yoloye, "Socio-economic Pee Ghana 


-economic background and Sc 
of children in three type 
Countries, vol. 12, no. 


; Routledge and Kegan Paul, 1965; 
i A survey of the backgroun 


State of Nige ia", The Education in New 
L, 1955. p. bap, geri ate Education in New 
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schools 4 Finally, there were some specific studies carried out in connextion with Apsp 2 

Some of the findings that are directly relevant to APSP include the following: general 
intellectual ability and manual dexterity are the most critical factors in determining favourable 
teacher response to the use of the APSP approach; previous knowledge of science is not 
absolutely essential for teachers to use the APSP approach; age and teaching experience correlated 
Positively with favourable teacher response to the use of the APSP approach; the reading abilities 
of the average primary school teacher are high enough for them to comprehend the majority of APSP 
unit teachers! guides; and the reading abilities of the average primary school child are such that 
the children may have difficulties with their background readers. 


Evaluation of transactions 
uation, the bulk of evaluation activities was concentrated 


Because of the emphasis on formative eval 
d techniques for transactions. 


On developing and trying out instruments an 


Evaluation of instructional materials 


Because of the open-ended nature of APSP approach, it is not always easy to identify precisely 
what learning experiences the units are supposed to be providing. One task of evaluation was 
therefore helping to identify what each unit is likely to do. Two approaches were used in 


determining the potential of the various units. 
a modification of Gagne's technique, sets out to identify as precisely 
a unit sets out to teach and the experimental operations, observations, 
and logical operations involved in the process of teaching thems <1) aitere E me me procequng 
in that it is carried out on the written units rather than as a foundation for preparing t i ae 
Strictly speaking, the technique is more suitable for structured, 'eoncept-oriented curricula than for 
the open-ended approach of APSP. However, APSP units vary in their degree of structure =, 
Concept orientation. Some units are sufficiently structured and pa es to be amenable 
to task analysis, but most APSP units would not lend themselves E sped pes den eet 

When applicable task analysis serves two main purposes: E. eps: : n e alert 
Sequencing of various experimental operations and Sporting sire cw sd é oer s m 
and thus helps in the writing exercises of a unit and it provides a basis tor etting 


Sissi ue h the processes of science: fifteen 

E j ia] of APSP units to teach the pro : 
(AAAS) Science-A Process Approach list - were selected. as cuts on pi PEL uh 
Panel of eight to ten "experts" in science on à dete i Sok Whencva Quia Whe Wide 
teaching each of the fifteen processes. Ratinen Nd ts h each rater showed reasons for his 
divergence in ratings, there was 8 thorough discussion in wae tof ratings. In addition, each 
Or her sating E ae discussions usually ended in closer agreement o ings. ; 


Task analysis: this process, 
as possible the concepts which 


: , (Mimeo) I.L.O. Interim report, 


D. Calcott, "Education in a rU 
Ministry of Education, Ibadan, 1968. 


ral area of western Nigeria" 


ME : ilie 

to training on APSP materials", Journal of 

E.A. yolo "A | teacher response I-A 
ie ge, tA abide o! respon? vg, no. 1, 1970, p. 21-25; E.O. Ogunyemi an 

Science Teachers Association of Nigeria io hysics achievers in two Nigerian 


high and low p! 
ier Etita, " Cognitive pretty and dT. Bettie, "Cognitive preferences in mathematics 
ondary schools" and E.O. Ogun! 


i in two Nigerian secondary schools", African Journal 
amo atics achievers in aay 
ot PoE bileh aha Ton meari 1 o Ly 1974, D. 107-114 E 97 a pon E i 
"Roadaiilite macer aike evaluation of curriculum materials", Journal of Curriculum es, 
y indices in UE 
vol. 7, no. 1, 1975, p. 78-84- 
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unit was rated as to its suitability for upper, middle or lower primary classes. Profiles were 
then constructed for each unit. 

On the basis of these profiles, a number of major observations emerged. All units analysed 
appeared appropriate in terms of difficulty and complexity for primary school children. The 
profiles of each unit could form a basis for selection of what units to combine for a curriculum 
So that the entire curriculum is balanced in terms of the processes catered for. 
do not seem to be catered for sufficiently by the nineteen units analysed. In particular, there is 
little or no indication of making operational definitions and formulating models. Others which are 
only scantily catered for include prediction, inference, formulating hypotheses and interpreting data. 


Some processes 


Readability of APSP materials 


Another kind of evaluation carried out on the APSP materials was directed at assessing how easy 
it could be for the audience (teachers and children) to comprehend the materials written for them. 


The Lorge formula for estimating the difficulty of reading materials was used for calculating 
the readability indices of thirty-seven APSP units. 


For comparison, the readability of ten 
Elementary Science Study (ESS) units and three Nuffield junior science materials were also 
calculated. 

Taken along with the data on reading ability of primary school teachers and children, the 
data on readability indices provided valuable insight into the probability of the materials being 
appropriate for the teachers and children. 

It was found that the teachers! guides were all within the range of comprehension of the 
majority of Grade II teachers. The most difficult unit has a readability index of 6.84 while the 
twenty-fifth percentile for Grade II teachers is 7.15. In addition, for the majority of primary 
schools, there may be difficulty with the Pupil materials. Five of the eight books analysed had 
readability indices of 4.83 or higher, At least half of primary 6 children in the sample tested 
(median reading level 4.73) would have difficulty with these materials. The mean readability 
indices of ESS and Nuffield materials analysed are 6.07 and 6.59 respectively as compared with 
5.38 for APSP materials. This indicates that the APSP writers were in fact sensitive to the 
reading abilities of their audience, 


Evaluation of instructional strategies 


Part II of the goals of APSP refers to tra i -Á 
ns in i i " 
Which may be summarized as follows; actions. They suggest certain instructional strateg 


child: i 

crine dics ane a a concrete materials than from the teacher. 
, ime i i WF 1 

concrete materials by the Hilaren Science class should consist of activity with 


children and small groups of children SS at most times should be through individual 


; to provide a basis for 
purpose of evaluating the programme. "Este: GF fie APOP: mpbronoh dus E 
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Evaluation of the African Primary Science Programme (APSP) 


Three instruments were developed for these purposes: 


The class interaction: this is used for recording objectively the pattern of interaction of 
the teacher with the whole class. It is a category system of direct observation making 


use of five categories: 


A. group or individual activity; 
B. whole class discussion; 

C. transition; 

D. teacher lectures; 

E. other. 


Records are made at one minute intervals although smaller intervals (e.g. thirty or fifteen 
Seconds) have also been used. At the end of the lesson, an interaction profile is drawn 
based on the number of entries in each category. i ; 

If APSP approach is being correctly interpreted, a profile showing the highest point 
on A followed by B but with very low values on C, D and E is expected. For teacher- 
training purposes, records of a teacher's performance either in micro-teaching situations 
or normal class situations can be used to monitor the teacher's progress towards the 
expected pattern. The profiles, coupled with sequential pattern of the entries, also 
provide fruitful bases for initiating discussion with the teacher. í ae 

For programme evaluation purposes, the instrument is useful for identifying 


"experimental" teachers whose classes will be compared with control ones. 


WE i bjectively the teacher's pattern 

T i i eet; this is designed to record obj teac 
Zhe verbal interaction Siea class. Much of the stimulation of the child is through the 
z r teen al statements could foster or hinder the creative 


ts. Such verb é Es : 
Dem x gue crit child. The verbal interaction sheet identifies seven categories 


of verbal statements, namely: 


direct instruction; 
open-ended question; 
close-ended question; 
suggestion; 

giving information; 
supportive action; 
comment implying th 


at the child's statement is unacceptable. 


i i hey occ 
Entries are made in the sequence in which they 


end of the lesson. mium on statements in categories B, D and F. 
T ic objectives of APSP place pre 
Tuer. «ro profile should show peaks at B; p and F 2 dea d Ta = pa E . 
re, lass interaction sheet, this instrumen can be | fo 
As in the case of the ` hing Or normal class situations , and for identifying teachers 
training purposes, in micro-teae zp erimental classes in programme evaluation. 


with appropriate verbal interactions for exP 


eussos» 


ur and a profile is constructed at the 


the need to let the child work at his or her own pace. 
between teacher and child, the teacher is often 

the child, with the result that slow-thinking children 
ral, become inhibited in their thinking processes 
ther question or statement before they have 


Waiting time sheet: APSP ampan s 
During question-and-answer exchange 


; 3 m 
impatient to have the right anaon ae 
in particular, and indeed children á g aieh anol 
as a result of the teacher apes ia 

had time to think through d es an objective record of how long a teacher waits after 

SN waiting ies pion again. The records also provide some evidence of the 

each question before Spe MEC her talk. 
extent to which children are inhibited by teacher 
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This instrument is specifically for teacher-training purposes. It is particularly 


i s 
ffective in micro-teaching situations for monitoring the progress of the teacher toward: 
= A P 
greater patience in handling children. 


Evaluation of supportive strategies 


i ols 
Supportive strategies include in-service courses or workshops, follow-up of teachers in scho 
ea i ople 

i action with policy-makers and resource pe p k ; 
Ps po neas in these respects focused on developing effective feedback systems so M is 
supportive strategies may be progressively improved. APSP used a variety of E ker] 
du purpose including: reactions to a training course; report on teaching of science units; 
materials information sheet. 


Evaluation of outcomes 


Two summative evaluation studies were carried out with r 
programme On children and teachers. 


Outcome on children: Duckworth ! 
zutcome on children; 


espect to some outcomes of the 


-APSP children on all tests. 


compared six APSP classes with Six non 
Sierra Leone on an interaction analysis schedule based on the Flanders b 
nsecutive lessons were Observed in each 


-APSP classes in 
ut having twenty 
class, with findings 


Five co 


7 in APSP classes, 


the incidence of Pupil acti 
whereas the rever 


vity was higher than teacher activity 
Se was the case in non 


-APSP classes; 
-  APSP teachers used predominantly indire 


ct influence whereas non-APSP teachers 
used predominantly direct influence; 
- Pupils in APSP classes exhibited more independent behaviour than non-APSP classes; 
and 


length of contact with APSP did not Seem directly related to the amount of change 
in teacher behaviour, 


Concluding remarks 


lgtésnor R. Duckworth, A c 


omparison study for evaluati: rimary school science in Africa, 
Newton, Mass., E.D.C., 1971. 


E.D.C., 1970. 


Evaluation of the African Primary Science Programme (APSP) 


In the course of APSP evaluation, it became clear that the crucial criterion for assessing 
the success or otherwise of APSP cannot be merely "whether or not the materials work in the 
African classroom" but more importantly "whether or not APSP has contributed significantly to the 
production of a core of people within Africa, with experience, expertise and a mental orientation 
geared towards continuous curriculum renewal". 

There is now a substantial core of such people all over Africa. The harnessing of the 
resources and expertise of this core of people for joint effort at tackling curriculum problems in 
various countries has been carried out in a most effective manner by the Science Education 
Programme for Africa (SEPA). - 

Not only has this organization promoted continuous training of manpower through two 
international institutions - The International Centre for Educational Evaluation (ICEE) at the 
University of Ibadan, Nigeria, and the Centre for Science Edueation in Njala University College, 
Sierra Leone; it has also made it possible on several occasions to assemble the best available 
human weacurces in Africa to work together on in-country programmes in curriculum development 
and evaluation. 


In fact, SEPA may be said to have evolved a unique model of educational development in 


African countries through its emphasis on manpower development and harnessing of expertise with 
a minimum of administrative staff. Since its inception in 1971, SEPA has organized over a dozen 


international workshops, training programmes and seminars all over Anglophone as well as 
Francophone Africa. 


ICEE has trained more than 
and has a current intake of ten fo 


thirty people in educational evaluation in the past three years 
r a one-year diploma. It has also conducted four major 
curriculum evaluation studies at national and international levels. The ee Centre 
in Njala has produced its first set of one-year diplom sa not been adopted wholesale in am 

In terms of the materials produced by APSP, they have not xe OD ale + ee y 
country. However, they have formed substantial inputs into nationa — e p a E A 
participating African countries including Nigeria, ‘Ghana; seg Sense Vi Een od Bus 
and Tanzania. The fact that no wholesale adoption has s ec à "M by Lm patie 
because ironically if the programme has produced mhe AUA eidem aho "i e been ? 
one test of its success would be that in the come indi nn Em va have had contact vd APSP 
completely replaced by new materials produced by the very People w k 
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ence Education, APSP Monograph Series, 


Robins, Ralph H., Programming for Change in Sci 


Newton, Mass., E.D.C., 1970. 
s .P., 1976. 
Yoloye, EscA us Evaluation for Innovation, Ibadan, 1.U 
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11 Summative evaluation of Caribbean 
integrated science projects 


dith F. Reay I ae 
eae of the West Indies, Trinidad 


SUMMARY 


" — r i d 
In the Caribbean in the early seventies, there was a complex situation in which several integrate 
Science innovations were taking place in the region. 


The major strategy for summative i 
evaluation was based on an educational experiment and upon achievement tests, but other forms O! 
evaluation have also been used. There is little evidence that the data were used by the 
decision-makers, and it is suggested that, in the end, decisions are taken on the basis of 
availability of published materials and individual appraisal of these. Teacher education courses 
Should pay more attention to the Significance of evaluation data. 


INTRODUCTION 


In planning for the evaluation of 


a curriculum development Project, one has to take decisions 
on the following questions: 


what is the purpose of the evaluation? 
what can (and Should) be evaluated? 
what procedures will be used? 


As Cohen points out in the intr 


will be heavily dependent on the co 
evaluation in a sett; 


Tyler! and Kerr? 


the answers to these questions 
This case Study describes an 
ing where the sort of neat curriculum development Strategies outlined by 


mter, Op. cit. 


2 
J.F. Kerr, * Currieulu 


m change in emergent countries" 1 
New Curriculum 


in G. Howson (ed), Developing a 
; London, Heinemann, 1970, 
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The purpose of the evaluation 


In the Caribbean in the early seventies, there was a rather complicated situation in science 
curriculum development for the lower secondary level. UWI at Trinidad was completing the 
development of WISCIP; UWI at Jamaica was continuing the development of a programme that 
had been begun by the National Curriculum Development Committee (the UWI phase being known 
as the MONA Project); and UWI at Barbados was the headquarters of the CEDO/UWI Caribbean 
Regional Science Project embracing fourteen countries of the region including Jamaica and Guyana 
(but not Trinidad and Tobago). Why were there three such projects, all co-ordinated by UWI? 
The explanation is that at one and the same time, the West Indies is a region and it is 
fifteen countries with fifteen Ministries of Education, a situation that must be familiar to the 
South Pacific and to Africa. The separate governments of the region established junior 
Secondary schools within a few years of one another so that secondary education would be 
available to more children than had been served by the selective grammar schools; it is no 


Coincidence that the curriculum development projects sprang up at about the same time. . 
ols are generally subject to considerable government direction, 


although some of their teachers are very ready to criticize the curriculum and suggest improvements. 
The grammar schools are much more autonomous, but many of them choose to use the same 
curricula as the junior secondaries, for good reasons. . : i 

Thus the iban and their junior secondary schools were very interested in the science 
: y of the grammar schools. Other schools, reasonably 


curriculu ts, as were man ols. — ) : 
Satisfied pi ap nap programmes and reluctant to offer their children as guinea pigs, were 


Cautious about participating in any project. a . s 
An siesta enini e the evaluation was the accountability to funding agencies and 


Bovernments However, the actual form of the major evaluation was probably determined by 
Jamaica li both MONA and the Regional curriculum were in trial. Thus an important — 
purpose be can. to help teachers, the government and the curriculum committee make a choice 


between the two curricula. 


The junior secondary scho 


What outcomes would be evaluated? 


i ts. Teachers can become more 
Curriculum i tion has a variety of possible effec j i 
aleae an a amit they are teaching and they can become more ipae at ips 
Children to : abou CHIE can acquire skill in the processes of science, they can gun F Y 
len ens exe sence better and they can develop attitudes such as self-esteem, understanding 
pts sei , 


i i hers can 
their environment. Children and teac 
m foncer fop: alone DNE and ia, Dore Parents and other members of the community 


become i ching-learning situation. à ipei 
can deed ¢ Utne neers of what science learning is all about and can be more willing 


i igations. 
Sor i i i i t-of-school investiga 
bes es — Se caepitiondlly difficult to evaluate, but all of them were goals of 
e o ese outcome 


i i d with cognitive 
the ; rs in the region at that ume were more concerne v 
tchievemon iar wi anying elso., A PORTH TOG) e. Mesures were mall 
examination results which it produced would be a eget E Mae a ee 
and time was short, so the project workers decided, ees y obe due b dae. cott 
o native 1 t i ould attempt to aie d oe th decision-maker; in the Caribbean 
(Chapter 2 [ve eae ed that the evaluator i$ the servant 9 4 ive Sessa y^ sessi yi 
exercise à ss any d ortant (in view of the gon EN vide rd sobres ance be 
evaluation psg ws ebony as possible and that would prov 


©cision-makers kw 


The procedures used 


Setting u 
p an experiment à i of the schools in 
Th „one Banque (CRSP was. importani, nique TT ly all th 
e Caribbean Regional Science ome of the schools in Guyana and Belize and nearly a e 
E 


arbados, ten schools in Jamaica, 
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dary schools in the smaller islands. The MONA! Project, which began at about the oY 
E i ae 1970), was much smaller, being only part of a science teaching improvement S . 
pte ss that Jamaica should set up a CRSP/MONA control experiment, oes - — 
the Jamaican CRSP trial schools would be a set of MONA schools as similar inco MR 
organisms can ever be. Because of the limited resources of the Jamaican project, t A 
set of schools (the controls) would not receive special attention until the time came for : m 
summative evaluation. This was not considered a serious weakness in the experimental A 
The improvement project had, after all, many activities and their influence on all schools wou 
be taken into account when selecting the controls. 


Communications with trial schools can be a problem; in Kenya's SSP and Guyana's SDSP, 
for example, trials have been restricted to schools close to the curriculum development centres. 
In Jamaica, for experimental as well as for educational reasons, it was thought important that 
the trial schools should cover a broad spectrum - rural and urban, fishing village and 
banana-growing areas, selective and non-selective. The projecfs resources were stretched 
rather thin by having trial schools in most corners of the country. The classification of the 
trial schools is shown in Table 19. 


Table 19. Classification of trial schools in Jamaican curriculum experiment. 


MONA CRSP 
Selective 2 3 
Non-selective - Comprehensive (poor facilities) - 2 
7 Junior secondaries 
World Bank (good facilities)* 5 
+ Converted (poor facilities) 1 
Total 8 10 


* Of these, three tried out both curricula in different streams. 


atched as carefully as is possible in an educational 
experiment, which, being concerned with 


experiment. Many of the variables could be taken into account when the summative evaluation 
was carried out. 


Preparing for summative evaluation 


The Caribbean 


about three years. While this 
leads to a cer 


“year curriculum has to be designed, - 
S and then finally published, dissemina 
erating curricula quickly. Commonly, 


It is common for funding of a project to be 


only three years at a tim Sion to renew, if it happens, always seems 


to be taken at the eleventh hour. 


1 
J.F. Reay and A.D. Turne 


T, New World Science. 
curriculum, ) 


(The published version of the MONA 
Pupils! books and teachers? 


6. 
guides, Trinidad, Longman Caribbean, 1975 and 197 
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With such short projects, it is necessary to conduct summative evaluation during the 
development stage. Data were collected about the performance of children after they had been 
following the courses for one and two years, but it was not possible to examine performance after 
a full three-year course. í f 

There was also the question of who should carry out the evaluation. Williams has commented 
that evaluation requires a certain detachment from the day-to-day detail of curricula. Tawney 
observed that, as far as summative evaluation is concerned, self-report has low credibility. The 
projects lacked the means to employ any evaluators; however, independent experts in the region 
were approached, It was hoped that one of them would take responsibility for analysing and 
interpreting the data. Perhaps because of the delicacy of the whole situation, this kind of help 
was refused, although the experts were very generous with advice. Thus the innovators were 
also the evaluators, which had the advantage of costing nothing. In the end, however, this may 
have reduced the influence of the exercise on decision-making. 

The instruments of evaluation presented a difficulty. Mayer (chapter 6) stresses that the 
quality of the information gained is only as good as the quality of the instrument, and argues the 
need for: better co-ordination in the development, repeated use and refinement of instruments. 

In the Caribbean, searches failed to turn up any instruments which would be suitable for West 
Indian children and which would measure specific cognitive skills. , As it later turned out, 
Williams? had been developing achievement tests for WISCIP in Trinidad and Tobago and 
developing analytical procedures for the responses. The potential of his work for testing a 
variety of skills on a large scale and with small resources is very significant. Had the results 
of his work been available at the time of evaluating the experiment, ee ee 
influenced the tests developed for the eastern and northern Caribbean. s it was, origin: 
instruments had to be constructed. Pre-testing them (a difficult and mre ares a 
Mayer says) was beyond the resources of the projects; but some post-test inspection was done 


3 
and the validity and reliability were thought acceptable". 


The sampl 
ples 
ad led to the establishment of an educational 


Although it was the situation in Jamaica that h WE i 
sin th tive evaluation embraced eight countries in seven of which the only 
» the summative f CRSP. (CRSP!s curriculum should be clarified here. A 
P : ersion was called WISCIP/B. Formative evaluation 


“perimental curriculum was that Fe 

es i P, its firs 3 i 

all one of THIS bras development, in regional writing workshops, of WISCIP/C. This 
ver the Caribbean led to e the curriculum published as WISC, West Indies Science 


Version was further modified to becom 


i in Curriculum Innovation, London, Commonwealth 
I.W. Williams, WISCIP: A Case Study in c 


Secretariat, 1973. 

A 3E, Wiliams, Regort:on die GERE ome | ^" C menit e sei a 
Swansea eiiis College, 1971 (Unpublished); zx seniors (Years 1 and 2), held in 
University of the West Indies, on the Er Dee ion (Unpublished). 
Trinidad in July 1971, Swansea, University College, 


1 WISCIP Ex 


rst Year of Two Science Pilot Projects 


ion of the Fi 
irr P.S. Adey, J-F. Reay and A.D. Turner, 


J.F., Reay and A.D. Turner, An Ev: ons dios 
9n Trial in Jamaica, Jamaica, » 1 Science Curric Pd a tha ‘Caribbean - Interim Report, 
2 Evaluation of ‘New Junior pe^ Caribbean Science gea gee ig ae Test 1972, 
nth UWI à .D. Turner, 

ieri. , 1973 use P and P.S. Adey, L. St. E. Tores m. oe fe x 
Ah Beatie £ Eos aise Secondary Science Curricula in the Caribbean e port, 

valuation of New Juni 

Jamaica, UWI, 1973 (mimeo). 
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Curriculum 1 The MONA curriculum has been sublished as New World Science, 1975.) 
u " 


From the experimental groups, equivalent samples were selected. The selection of controls 
presented no problem in Jamaica, where most of the Schools were not 


participating in these 
innovations, although some were using other new curricula (e.g. Nuffield). In Jamaica, therefore, 
a respectable battery of controls faced the project schools. 


Barbados had rather more difficulty 
in selecting control schools and no suitable ones could be found in the other six countries. 
The costs 


When the results were processed by computer (1972 and 1973), the charge for the UWI computer 
service was of the order of (US)$100 each time. 


Materials costs were reduced by re-using the 
test papers, but in any case such expenses as paper and travelling, which are part of the normal 
innovation process, were not charged Separately to evaluation. Hence, the only costs specific to 
the evaluation process were those of the computer. 


The tests and their administration 


During the lives of the projects (1970-1973), 


three waves of testing took place. 
1971. MONA and WISCIP/B in Jamaica. 


An objective test was constructed with small sections 
to test a variety of skills. From precise instructions, teachers had the papers typed and marked 
the responses. The only change they could make to the items was to select a stated number from 
the recall sections to suit their Own courses. The whole procedure lacked some rigour in 
Selection, presentation to children, a 


nd marking, but analysis revealed no Serious weaknesses. 
The test was administered to 2,283 Form 1c 
were WISCIP/B pupils and 962 were 


hildren of whom 710 were MONA pupils, 611 
controls. 
We shall compare the 


performances of these three groups later 
other interesting findings 2. As with all subsequent 
better than those in non-selec 


t tests, children in selective Schools performed 
tive (comprehensive and junior 
otherwise, the instrument wo 


uld have proved invalid. 


P.8. Adey (ed), West Indies Science Curriculum, London Heinemann, 1976 


Reay and Turner, An evaluation » Op. cit 
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1972. MONA and WISCIP/B in eight countries. The same wave of children, now Form 2, was 
examined. Experience with the 1971 testing had suggested that tighter procedures were needed. 


A draft test paper, prepared by the headquarters teams, was shredded, modified and 
reclassified by independent and highly qualified experts. The categories of cognitive skills to be 
tested separately were Comprehension, Application and Higher Abilities, and the headquarters 
teams were confident that knowledge components had been minimized. Once more, parallel groups 
of children were tested, as shown in Table 20. 


Table 20. Samples for 1972 summative evaluation by Jamaican science project and Caribbean 


regional project. 


MONA WISCIP/B CONTROL TOTAL 
ee ae LL D D 
Jamaica 371 470 485 1,326 
Barbados - 364 375 739 

371 1,575 860 2,806 


ulum workers themselves, according to carefully 
n mark-sense computer cards. The human 
inated. Performances are 


The test was administered by the wR 
agreed procedures, and children responde o à : 
variability in administration and marking was thus virtually elim 
compared later (in Table 21). 
in eight countries. It was now time to evaluate CRSP's 
2878. WISCIP/B and WISIN EC The same instrument as in 1972 was applied in the 


modification of WISCIP/B to WISCIP/C. On this occasion, it was 


a ration of Form 2 children. 
crues Na E te he are at: selected in such a way that data could be compared with 


WISCIP/B and control results of the previous year. 


The information obtained 


After the first year of the innova 
MONA children performed better 
Performance as the controls, and 

However, with the same wave 


i 1 showed that non-selective 
i in Jamaica, Reay and Turner^ S 
e E controls , selective MONA children gave the same 
WISCIP/B children performed worse than the controle 
of children a year later, Adey, Reay and Turner“ showed, 


i of project children performed better than the 
inter ali i ica and Barbados all groups À 3 P/B i b 
ten fe haud toqer oai qe children (MONA in Jamaica and WISCIP/B in Barbados) 

, ept for 


i trols. 
Whose performance was the MEE as that no had followed WISCIP/C) were tested, their 
When the next wave of children o. 


Performa: better than Or equal to performance with WISCIP/B, except for non-selective 
nce was better 


was worse". 

Children in Jamaica and St. Lucia, koe pedi bles 21 and 22, which summarize 
1 n 
From these three studies, it has bee 


ble to compile Tal 
Some of the results. 


1 it. 
Reay and Turner, An evaluation -> ops 30h 
2 . cit. 

Adey, Reay and Turner, An evaluation -> s 


3 T 
Adey, Jones and Turner, OP. 9^ 
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" tadial 
i i tency revealed by this longitudina 
the workers at the time is the lack of consis itudix s 
bg pem year at a time, "hard facts" were collected at respectable levels of be ei M 
nm one puts them all together and looks back with the increased objectivity that comes ru 
ey lapse, it is difficult to maintain that anything has been proved. The tables certainly S 


i ts 
more pluses than minuses, but there is a lack of consistency from one year to the next that cas 
serious doubts on the validity of the results. 


Table 21. Comparison of mean scores on achievement tests: trial children against controls. 


aae ola 


1971 1972 1973 


eeo E a O O BEP O ced 


MONA (Jamaica) - selective 
- non-selective 


WISCIP/B - selective (Jamaica) 

- selective (Barbados) 

- non-selective (Jamaica) 

- non-selective (Barbados) 
WISCIP/C - selective (Jamaica) 

- selective (Barbados) 
non-selective (Jamaica) 


non-selective (Barbados) 


q——————————— c n RNNNENNRMM 
* better than controls 


worse than controls 
no significant difference from controls 


+ 
g 


nett 


++ 


. 
Table 22. Comparison of mean Scores o 


n achievement test; WISCIP/C (1973) children against 
= . WISCIP/B children (1972). 


EE e Lm 


Selective Non-selective 
Antigua - 
Barbados - : 
Dominica * i 
Grenada = " 
P RR B no results 
St. Kitts/Nevis none A 
St. Lucia + : 
St. Vincent 


3 a 
di 
*  WISCIP/C better than WISCIP/B 


-  WISCIP/C worse than WISCIP/B 
no significant difference between WISCIP/B and WISCIP/C 


One might ask if cognitive develo ng 
pment can be m i ing for 
one or two hours. One might go further and nées f wipe d du nlsi dast mag 
leave (and most Jamaican oul 


h events as teachers! maternity 
teachers are women a S teachers ; aw 
laboratories to schedule, or even an inportant oe r overnment's failure to complete its n 
influential than the curriculum. of cricket m 


atches, might be more 
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Other summative evaluation procedures 


Tawney (chapter 2) suggests that triangulation (the use of several methods) can improve the 
reliability of the information obtained, and Cohen (chapter 1) has outlined several possible methods. 
Some postgraduate students at UWI in Trinidad have been using the test developed by Adey, 
Reay and Turner! in small studies which compare the performances of Trinidad WISCIP children, 

Trinidad control children and other groups of children in the Caribbean; but no important 


information has emerged. 
Williams? has been analysing test scores of Trinidad WISCIP children and has found that 


Some curriculum objectives are apparently not being achieved. He casts doubt on the objectives 
themselves (Tawney, in chapter 2 of this book, has pointed out that one must think seriously 
about this) and observes that a number of very important effects of the curriculum are not being 
evaluated by achievement tests. Indeed, he observes: "I do not think it is possible yet, and 
certainly not on the basis of test scores alone, to evaluate whether the Project is successful 


or not", 

Yet another form of evaluation ( 
School) was carried out by a postgra 
children are continuing science after 
and their WISCIP grades correlate we 
are important to teachers in the Caribbean. 


inconclusive because it was done on a small sample in one 
duate student at UWI, Trinidad. Ishmael? showed that more 
being exposed to WISCIP, their GCE performance is better 
ll with their GCE grades. These are considerations which 


The author is currently engaged in a survey in the Caribbean to determine the extent to which 
the; innovatto boina used several years after the official end of the projects. It appears, 
ions are being , that in the small eastern Caribbean 


j i ilable 
just before any published materials have become avai š i 
countries VS dissemination base remains for WISC) most schools are using WISC; 


i i is tremendous. 
otherwi i rricula being used in schools is 
epoploccli gun M Seti Curriculum Committee and to CRSP, Reay and Turner 4 analysed 


the content and treatment of major concepts in MONA and WISCIP/C, this being an intrinsic 


evaluati i wake 

he ae for postgraduate students at UWI Trinidad, students were asked 
to make a 1 f the units of Trinidad WISCIP and WISCIP/C. It is almost certain A 
that the eene gen ede from one unit to another, depending on the consultants ane in 
them, but another determinant of preference appeared to be the degree of authoritarianism © ES 
Student: ienced graduate teachers)- " 

ra ong ced po sl. ed the WISCIPS but not a comparative one, was made by 
itis e careful app in which they developed a check-list for the evaluation of 
““acher-educators at a workshop in E ichever WISCIP they knew best >. 
integrated science curricula, and applied as to, une 


i i . cit. 
Adey, Reay and Turner, The Caribbean —' op 


cit. and Report to Institute ..., 


jan Science Curriculum Innovation Project 
d, UWI, 1975 (Unpublished). 


2 op. cit. 
Williams, Report on the ..-: op. 


valuatioi : 
High School, Tri 


3 West Ind 
M. Ishmael, A Summative E n of idi 


(WISCIP) at Naparima Girls' 
ng of Science in Jamaica, Grades 7-9. Report to 


mmittee and to CEDO/UWI Caribbean Regional 


4 : 
J.F. Reay and A.D. Turner, The Teachi 
d science curricula, Jamaica, UWI, 1973, (mimeo). 


i -co 
Jamaica National Curriculum Science ud 
Science Project, based on trials of integ 
5 cajn, in Integrated Science and Teacher Education, Trinidad, 
J.F. Reay (ed), "Project Appraisal", 


UWI/Unesco, 1974. 
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Conclusions 


Beeby 1 has said "In my experience, the average teacher has a very great capacity for going on 
doing the same thing under a different name". The teacher, given a curriculum guide on 
The Gases of the Atmosphere (05, CO,, N,, etc.), is very likely to add all he knows about 
HS SO, and perhaps other gases. Evaluation often presumes innocently that the teachers will 
985 

use the curriculum guides as was intended by the headquarters team. Many of them will do 80; 
but a significant number of these will drop the curriculum after one or two years. The rest will 
never have taught the curriculum as was intended. 

Did the information provided by the evaluators help in the decisio 


not. Teachers seemed to base their decisions on whether the 
It appeared that the other decision 


and in the end, Jamaica's curricu 


n-making process? Probably 


y liked the look of the guides or not. 
-makers did not find any of the information compelling enough; 
lum committee made no choice at all. The most important 
information sought by the decision-makers appeared to be the kind that would be provided by 
publishers - would they publish and when would the publications appear? The availability of books 
for children seemed more interesting than anything else and the Caribbean evaluations apparently 
provided only academic interest. 


Would different strategies have proved any more useful? It is doubtful whether different 
techniques, or evaluation of different outcomes, would have fared any better. It is possible that 
a greater involvement of teachers, as associations, would have attracted greater attention to the 
evaluation results. At that time, the Science teachers! associations of the region were relatively 
weak; now, with more of them, a regional association and with national and regional associations 
compelling respect d on close consultation and assistance 
from the associati One still feels, however, that the information 
which teachers will want, will not be of the statistical kind obtained from children but data on how 
be ready. The determined individuality of teachers 
to recognize. 

-making should be based on valid and reliable evaluation - 
data; perhaps teacher education should pa: 


y more attention to convincing teachers of this and 
Should help them evaluate the data they receive. "a 


C.E. Beeby, "Curriculum planning", i 
E ; in Howso 
(p. 39-52), London, Heinemann, 19 inis 


70 (ed), Developing a New Curriculum, 
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onwealth Secretariat and has subsequently been 

to publish an abridged version, but may be able 


This report was commissioned by the Comm 
Curriculum developers with small 


brought up to date. The Secretariat hopes to pu 
to make the full text available to interested individuals. 


resources will find it most useful. 
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12 Case Study A: Evaluation of the Scottish 
Integrated Science Syllabus 


A. W. Jeffrey 
Her Majesty’s Inspectorate, Scotland 


SUMMARY 


This case study outlines the development and evaluation of a course of study 


in integrated science 
otland. 


It describes how the principle of 
aluative procedures which were used 


years of general science before splitting 

up into the separate disciplines for pres ion i ternal examinations. With the success 
of alternative syllabuses introduced in 1962 in '0' for academic pupils, 
it became desirable to produce a generalized a for the early s dary years 
using similar content, equipment and methods. Y En ry ose by 
the Secretary of State for Scotland i x Hs purp 

Shortly after this, a national 
along comprehensive lines. 


A 
n December 196. 


Working within a small budget 
procedures normal today. 
Curriculum Paper 7 - "Soie 


its headmaster and staff, Nevertheless, 
national level and is seen as continuous 
under continuous review and course mo 


are prepared to accept. Asa result, evaluation is also ced, hopefully, 
formative and summative Steps. 


1 scottish Education Department, Curriculum Paper T: 


HMSO, 1969. Science for General Education, Edinburgh: 
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Reflective evaluation 


During 1965, general aims were stated and restated. Course content was produced in parallel 
and judged on its potential to fulfil these general aims. The traditional lower secondary science 
course had been totally content-based but the working party decided to seek more process-based 
material which would at the same time deal with concepts relevant to everyday life and be within 


the pupils' comprehension. 
Three major constraints affected decisions. Problems of cost dictated that the content 
h laboratory equipment known to be already in schools. The 


Selected should be accomplished wit 
Examinations Board has to be satisfied that the content used could be examinable at 'O' grade 
(that is, an external examination administered at 16 or over after four years of secondary education) 


and was a suitable base from which their examination syllabuses in physics, chemistry and biology 
could be taught in the succeeding two years. This and the knowledge that not all teachers would 


be willing to teach integrated science required,a fairly even balance of physics, chemistry and 
d out by working party members and subject 


biology. During this time, evaluation was carrie 
panels of the Examination Board on 2 subjective basis. The chosen content was finally manipulated 
into a form suitable for trial. r 

including some affective objectives, was 


et of summative objectives, 
prepared and evaluated and restructured by the working party. It was decided to use the 
language of Bloom's Taxonomy t in stating these. In 1965, this was the first attempt ever made 
in Scotland to indicate that education in science might be concerned with attitudes, and in 
hindsight the working party was probably too timid, although it seemed at the time a very 


courageous step. 


Using this content, a S 


Formative evaluation . 
the material with their own classes and re-wrote them as a 


result of the classroom trials. The first full trial began, however, during cp nadar session 
in thirty-six schools, selected for their proximity to one or d a of the working party 
and thus easy access to advice as well as convenience for eva potu a teal PLATES 
During this time, feedback was mainly oral obtained "i mt ti 4 a a à to 
monthly intervals. The points generally discussed included: tl d reac is Pu ducta bo 
materials in trial the reaction of pupils to materials; the pro sagt js pregio me 
equipment; the guodesE or failure in teaching content and cats - e tim 
work under trial; and the results of internal tests on materia. aug m d S Rosso iacetnation 
The five HM Inspectors of Schools on the working party IN is i of il/teach 
at a more detached level. In their visits to schools, p pcm penes mom wins n 
Pupil/material and teacher/material interaction. They atso sii gone equally wailable t2 O 
above to moderate the feedback from teaches M iden AEA equally aoa 
evaluation, nor was the information provided by ated, it was rewritten for a second 


É 3 section was coll 
Ave information: akont ue gn ee the number of schools increased until 130 were 
m 


trial. This was repeated twice 

involved in the final trial. J d Se 
Several important things hie Er. Up became clear that this was too soon to tackle 

Secti Lo be s of energy- * i iffi 

Heim 9 lot s frm te eee scion 7 Pi hare were eed 

in constru bs pue s inhibi learning in i gui duced to the more simpl 

: natrno ing circuits wa. e original draft was reduced to e m re simple 

© minimize this difficulty. Ohm! because the conceptual difficulties were too great 


In writing sections, authors tried 


ction 2 of the first draft was Energy and 


i It. A section on space travel bss UO a to be taught in the final two of a six-term 
or qui i the origina’, > i hout. 
m pu om io and restructuring spread biology more evenly throughou 


vol. 1 and 2, Longmans, 1956. 


i jectives, 
1 Bloom at al, Taxcnon? of Educational Objective: 
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It was then also that worksheets were suggested and first drafts produced. Initially they 
were seen as a guide to the non-specialist teacher indicating sequence, depth of cover and 
processes to be learned as much as a guide to the pupil. 


Feedback showed that our estimate of 
the child's vocabulary was quite wrong and subsequent rewrites tried to reduce it to a more suitable 


level. It was thought by the time of general publication in 1969 that they were intelligible to all 

who could read but this has proved to be still too optimistic and they are again being revised. 
Another thing which emerged was the lack of skill among teachers in constructing suitable 

test materials. Because of our use of Bloom's Taxonomy, teachers were familiar with the 

increasing complexity involved in recall, comprehension, application and so on, but many were E 

unable to write test items to discover whether pupils were learning at these levels. In consequence 

a national training programme on objective testing was launched among science teachers during the 


subsequent four years and it was decided to produce and administer centrally-prepared tests. 
Only a small bank of items could be produced because of cost. 


Again, with hindsight, this was 
not good policy but the working party had a very small budget and little could be done at the time 
to augment it. 

One test was produced by Chaplin! on the first four sections o 
numbers of recall, comprehension, application and hi 
Bloom Taxonomy used by the working party). It was administered to a sample of 1,100 pupils 


in the trial schools. The verbal reasoning quotient (VRQ) of each pupil in the sample was also 
obtained from the school, this being the only measure 


of general ability readily available. The 
results of the four section test are shown in Table 23. 


Table 23. 


f the syllabus, with equal 
ghest ability items (this was the modified 


Student results on first four Sections of Scottish Syllabus. 
S 


ection tested Results (expressed as a percentage) 


Sub-test A (11 items at level of recall) 


46.8 
Sub-test B (11 items at level of comprehension) 43.3 
Sub-test C (11 items at level of application) 51.8 
Sub-test D (11 items at level of highest abilities) 50.8 
Over-all mean 48.06 


From this it was concluded that 


Pupils were ], i i 
for which items had been prepared. PD meguataly af dt a Ge 
Since the working party 


was also concerned about wheth i ili a 
$ 2 er pupils of al bility wer 
ae adequately, an alternative breakdown of the test results cà Referer ©" free groups 
pupils were identified according to VRQ, (i) » 100. (ii) 90-100 iii i j 
performance of each group on "eur imo 


each sub-t i i 
Figure 13, est was then determined. The results are shown in 


J. Chaplin, unpublished material held by the Scottish Education Department 
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Figure 13. Performance of subgroups of pupils with varying verbal reasoning quotients. 


Sub-test A  Sub-test B Sub-test C Sub-test D 
Recall Comprehension Application Highest Abilities 
8 
7 
6.5 VRQ ^ 100 N-720 
Average 
Sub-test 6 
Score Bis 
(out of 11) x 5.0 VRQ 90-100 N-292 
4.5 4-4 VRQ < 90 N-88 
4.3 4.1 
4 
3 


m this that all levels of ability were learning from the work they were 

e success of mixed ability teaching. ^ Various investigations 
lied to a group of over 500 randomly selected pupils produce 
national groups. These results could therefore be taken 


It was concluded fro 
doing. This was encouraging for th 
have shown that in Scotland, tests app 


results very close to those obtained from 
BS indicati ational performance. 
reasonably indicative of nati 7 on electricity was administered to 400 pupils, fifty 


on section 
ecole n tratified to represent the different organizations of 


from each of eight schools. Samples were S : nt c 
Streamed and rtwenad selective and unselective groupings. The results indicated that 
Streaming had little effect in improving the performance of the more able but had a considerable 


and lowering effect on the least able. 


Summative evaluation A " 
ed in the trials and the curriculum paper in preparation for 


form of summative evaluation. Money was still in 
test on the full first-year course. This was 
position to that used for his test on section 1-4 


By late 1968, with 130 schools involv 
Publication, it was decided to begin some 
Short supply. ^ Chaplin produced 4 gatem 
administered to a sample similar in size and com 
e: E ien ken poe King! undertook to assess the cognitive achievement in the 
dac EE m patr and administered tests on each of ede ce Ar oo 
of the dimbaliy eim schools had in finishing the haa feats fori tne iui fexens Sennen’ 
were not always administered to the mede. ie oe pe in performance: hetween 

King concluded that, although H a all levels of pupils was still reasonable and sufficient 
more and less able groups, the lear d It is doubtful if the evidence really 


in use. 
a nd paige Bed qe gms e, ING that they were too optimistic. 
loce nns ped sic t te about the inclusion of the affective objectives in 
During 1970, there was const 


able deba : ji 
t À sen also clear from visits paid to schools by HM Inspectorate 
he general statement of intent. ting to teach towards attitudinal objectives. In consequence, 
ee many teachers were not cong east available for the first time to mount a properly 
lt was fortunate that larger sum: 


y became i 
funded and Brown undertook the work, as reported in the next chapter. 
ed evaluation programme, 


3 is, University of Glasgow, 1970. 


J. King, Unpublished M.Ed. Thes 
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i itten as a brief non-technical statement of the place of science 

oe te form which that education should take in the early years of a 
— = E pde System. It was therefore inadequate as a basis for the more detailed 
pus e Re ce In any similar document in the future, it may be that the theoretical 
ee structure and objectives of the recommended syllabuses should be included. 
argum! 


i 1 sees other reasons for teachers not following the recommendations of Curriculum paper 7. 
ae tudy shows that children, howéver taught, are less enamoured of science after two years 
= penso io than they were at the beginning. ^ What can be done about this must 
ub. all of us for some time to come. 


Second phase evaluation 


In Scotland, each school head and staff are 
consequently for the various syllabuses whic 
in 1971-72, an investigation on survey of e 
that, by 1972, somewhere between 65 to 7 
first year. What was also true and less 
objectives. In consequence, during the 1971 
HM Inspectorate. Inspectors visited and 
Schools using the syllabus. 


collectively res, 
h form that curriculum. 


alive stage - Some six ^ ups 
Spread across the country - seeking wider diffusion of th j ten working group 
co-ordinate the work of thi 


S towards achievi 
During 1974-75, trials of thi. ee gs 
on the achievements of the least 
group can be established, 
easily determined. 


ction and the achi 
erent schools and for us 


E ent f " ^ ives can 
different abili 1 of the Various objective 
under the direction of Kellington 3, 


" t 
els. This work is being carried OU 


1 Hamilton Reid and Walker (ed), C 


ase Studies in Currj 
Routledge, Kegan and Paul, 1975. Srriculum Cha; ©: Great Britain and USA, 
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It has frequently been said in Scotland that the teaching of an integrated science programme in 
the early years inhibits the proper growth of learning in chemistry or physics or biology afterwards. 
In order to investigate this claim, the performance of candidates at the 'O' grade (fourth year) 

In 1970 about 20 per cent of pupils could be 


examinations is monitored from time to time. 
in 1974 it was nearer 70 per cent. 


expected to have experience of the integrated science course; 


National examinations 
The objective tests used at national level 


A sample of such monitoring is shown in Table 24. 
A new FV has 


have so far been kept confidential and items have been used more than once. 
been calculated for each re-use of each item. 


Table 24. — National examination results in Scottish integrated science. 
Pre-test First use Second use 
o Uo mee 1/2408... 
Item Year FV Tb Year EV Tb Year FV 21 
1 1972 .75 .56 1975 0.80 0.54 
2 1970 0.68 0.44 1972 0.57 0.58 1975 0.62 0.62 
0. 0 
3 1970 0.40 0.43 1971 0.30 0.32 1975 34 v 
z 1975 0.65 0.61 
4 1970 0.57 0.39 1972 0.61 0.55 
: 1975 0.61 0.56 
5 1970 0.60 0.47 1973 0.66 0.54 
.59 1975 0.52 0.59 
6 1970 0.43 0.62 1971 0.56 0 = e 
0.45 1975 : .5 
T 1970 0.62 0.54 1971 0.66 
72 0.66 0.62 1975 0.62 0.65 
8 1970 0.53 0.48 19 
972 0.76 0.66 1975 0.70 0.65 
9 1970 0.70 0.57 1 : 
972 0.76 0.36 1975 0.74 0.42 
10 1970 0.77 0.29 1 


FV - Facility value 


Tp - Point biserial correlation 
has been no change in the general standard of performance 


this evidence lies in the fact that during the period 
d in physics and chemistry have increased by 


. The facility values show that there ; 
in the examination. The significance © xamine! 
covered by the investigation, the Phe k a large number of candidates previously considered 
about is increase con oted that during this 
table to cope wi te difci level ot e ARES loss dun 10 percent nn 
á und oup a i : 
"ime the tota] number of pupils egi c other facets have been pun mes cus a 
i i integra. i 
$us During the life of the m n. investigating school-based responses to ees d n 3 
: dividuals or groups. Hamilto É trying to introduce the integrated science rs us. ss 
t departments of two pem about the: quality of our original M aR e ges out 
eful information has been obtai intention and practice. Roebuck t, investigating the 
the attenuation of objectives between s in remote schools (PHI) used sections of the Integrated 
s 


Potential of self-instructional materia 


1 €— Materials in Remote Secondary Schools in Highlands 
Roebuck et al, Project PHI Pr ty of Glasgow, 1974. 


and Islands of Scotland, Universi 
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Science syllabus because of the related specific objectives. 


From PHI a great deal was learned 
about alternative modes of presentation, including a game which improved comprehension of 
electrical circuitry. 


In this necessarily brief report of a decade of activity, it has only been possible to outline 
the efforts of numerous dedicated people. 


Probably some of the things done in the early days 
would now have been done differently. Others who may try to follow can learn from our 
mistakes as, certainly, we have done. Most of what we have done appeared reasonable in the 
circumstances. 


Footnote: I would like to acknowledge publicly all those who have given so willingly of their 
time and their expertise for these activities. 
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13 Case Study B: Evaluation of 


attitude objectives 


md Brown 
niversity of Stirling, Scotland 


SUMMARY 
This 
c 
he Fa study exemplifies evaluation of the attitudinal aspect of integrated science education 
or aries is from two perspectives: firstly, the purposes, meanings and feasibility of a R 
e objectives are examined, and secondly, the achievement of these objectives by pupils n 


and 
the factors influencing that achievement are investigated. 


I 
NTRODUCTION 
objectives in integrated science courses 


Altho 
ugh there ha i i 
ve been increasin 
for evaluation of this aspect of the curriculum have not received 
to provide such exemplification. 


exam 
Se ers of strategies appropriate 
Th ttention in the literature. This case study attempts 
- e study, financed by the Scottish Education Department, centred on five attitude objectives 
proposed Ty chain im pio fret = condary school in Scotland. The intention was that 
Pupils should acquire: 


g emphases 0n Mum 


o years of se! 


hip of the different disciplines of science, 


A]. 

A2, awareness of the inter-relations 

a : awareness of the relationship of science to other aspects of the (school) curriculum, 
3. i cho 

A awareness of the contribution of science tO the social and economic Titexofi fharcomimiidty, 
4. ; 

AS interest and enjoyment in science, , 

l ation and in assessing observations. 

ns, the first largely non-empirical, 


an objectivity in observ 


tualized 2S two broad questio! 


Th 
the se e evaluation was concep 
cond empirical: 


1, 
Are the arguments presented for 


are the " oses n 
meanings and PUrPO® | on exhibit? 


objectives educationally valid, 


the attitude 
To what extent do they 


what factors influence that 


5 correspond to attitudes that chi the pupils? 
i e 
To what extent are these objectives achieved by P 
Th achievement? 
e non. 
n-empirical study iticism' (analago 
* eva] ne of icurriculum Cri ipie ( guns, gathupe, to 
ra. uator's task here was seen as © al to the audience (e.g. science teachers) 
the "Y criticism) whereby the critic atem "e curriculum. 
eanings of the various ideas ! dd 
H 
op. cit» p. 16- 


Soo. 
tti 
ish Education Department, 
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The arguments for including attitude objectives in the course 


The course document CP7 (Curriculum Paper 7, Scottish Education Department, 1969) was examined 


for justifications for inclusion of these objectives. This was done from three perspectives. 


Firstly, a search was made for direct arguments for emphasis on affective goals. 
such arguments as they occur in the literature can be grouped into seven categories. 
concept of the ideal society and good citizens; 


In general, 
First, the 
assertions are made about the characteristics of 
an ideal society, the 'good' citizen's role and appropriate attitude objectives for the production of 
such individuals. The next category involves desirable social values and behaviours: —— 
Objectives are selected to produce 'good-citizen' qualities for Society as it is, rather than ior e 
ideal. Pupils' social adjustment regarded objectives as a means towards fulfilment of pupils 
needs for social adjustment. Another category was rational, intelligent, valuing behaviour: 


attitude objectives are instrumental to the achievement of such behaviour. Attitudes conducive to 
cognitive functioning were yet another affective goal; 


attitudes (e.g. objectivity) are valuable in 
cognitive tasks of scientific enquiry. 


Then there were affective routes to cognitive goals: 


for example, interest and satisfaction are means to improved cognitive performance. Finally, 
meeting pupil needs (other than social adjustment): 


objectives relate to pupil needs for satisfaction, 
enjoyment, etc. 

CP7 provides no such arguments for inclusion of its general attitude objectives. | However, 
the nature of the specific objectives for each section of work perhaps implies arguments of types 
involving cognitive functioning and cognitive goals. 

The second approach looked at the particular meanings ascribed to ‘integration’ by this - 
curriculum, and those attitudes logically related to such meanings. The meanings of ‘integration 
may be classified into three groups. Firstly, unity of all knowledge: knowledge is seen as one 
conceptual structure and/or enquiry process. Secondly, unity of the disciplines of science: 

Science is seen as one conceptual structure and/or enquiry process distinct from others and without 
component disciplines. Thirdly, interdisciplinary study: science is seen as the study of topics 
from viewpoints of various disciplines. 

Integration in the first sense wou 


The logical 
-relationships among the 
implies no particular attitudes towards science 


n used in CP7, though there is some lack of 
cument and the curriculum materials in that the 
organization of content in the latter reflects only 'interdisciplinary study'. In so far as the first 
Sense is accepted, there is justification for inclusion of the first three attitude objectives 
(Al, A2, A3.) 

The third possible source of justifications was the rationale for the integrated course itself. 
Arguments for presenting science in an integrated form are generally in terms of anticipated 
ts and demands under which the course must be 


or demand achievement of attitude objectives will have 
ch objectives. 


The types of arguments found in 
grouped in the following six categorie 


1. 


the literature for the value of an integrated approach may Þe 
8: 
Outcomes demanded by 


Society, 
political leadership = 


Rois i d 
©-8- provision of scientists, informed lay population, informe 
Resource constraints, e.g. accommodation, equipment, time, teachers. 

Political demands, e.g. common-core course for all pupils, national assessment system. 
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4. Conditions for effective learning, e.g. pupil security, motivation, interest. 
5. Conditions for effective teaching, e.g. teachers' interests, competence. 


6. Demands imposed by the subject, e.g. unified nature of scientific enquiry. 


Types 2 and 5 (concerned with resource constraints and advantages for teachers) do not imply 
attitude objectives for pupils. Type 1 suggests attitude objectives concerned with the broad 
influences of science on society; type 3 might (if the 'demand' were 'mixed ability groups 
following a common course!) dictate the development of attitudes that imply science as involving 
the whole community, type 4 may imply particular attitudes (e.g. interest) seen as related to pupil 
leárning; type 6 will imply attitudes related to the particular meaning of 'integration' used, and 
those attitudes (such as objectivity and scepticism) assumed to be characteristic of scientists. 

CP7 provides arguments for 'integration! of type 1 implying objective A3 (social and economic 
implications), type 3 implying objective A3, type 4 implying objective A4 (interest and enjoyment) 
and type 6 implying objective A5 (objectivity) and, in so far as 'integration! means the 'unity of 
all knowledge', Al (inter-relationships among sciences), A2 (relationship of science to other 


subjects) and A3. 


Clarity of the objectives 

CP7 provides no theoretical framework to clarify the purposes and meanings of its objectives. 
The objectives were therefore compared with a framework mapped out by the subject-independent 
categories of the Affective Taxonomy? together with Klopfer's^ categories of affective behaviours 
in science. Figure 14 outlines the broad correspondence between these two groups of categories. 
Comparison with this framewor bjectives for which further specification or 


k identified those ol 
ificati sary. 
War epe eoo readily related to Krathwohl's lowest level ‘Awareness’. 
However, lack of description of the cognitive behaviours that the writers had in mind for these 
objectives prevented unambiguous location on Klopfer's dimension. pon example, there was no 
indication within the course materials about what sorts of relationships between the sciences the 
i re of. 
bw ty eee ormani) covers three Klopfer (H4, H5 and H6) and three Krathwohl à; 
categories. This suggests that closer specification of the level of affective behaviour is required. 
A5 ('objectivity!) is identified with Klopfer's H3 category and. so covers three Krathwohl levels. 
It is not clear what level of commitment to ‘scientific attitudes is expected from this age-group. 
It is surprising that CP7 has no referents for Klopfer's first three categories: Nevertheless, 
if that gap were filled, necessary cognitive and affective behaviours specified and a level of 
commitment identified, these d form a comprehensive list which might be 


affective objectives coul 
related more readily to classroom procedures. 


pupils' attitudes 

respond to independent attitude dimensions exhibited by pupils? 
ves were developed . Items for these scales were 
e submitted to eight ‘experts’ who judged whether 

) to one of the five objectives. ' Position 


Relationships of attitude objectives to 


To what extent do these objectives cor 


Five attitude scales corresponding to the oa 
collected from conversations with pupils an avel 
or not each item related (positively OF nego 


f Educational Objectives Handbook II: 


^ m and B.B. Masia, Taxonomy O 


loo 
D.R. Krathwohl, B.S. B 1964. 


Affective Domain, London, Longmans, 
", op. cit. 


? Klopfer, "Evaluation of Learning -+ 


3.4. Brown and T.N. Davis "The p 
olds", Scottish Educational Studies, V 


elopment of an Attitude to Science Scale for 12 to 14 year 
l. 5, no. 2, 1973, p. 85-94. 
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Figure 14. 


Attitudes and interest 


Orientation 
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1.2 


1.3 


1.4 


1.5 


Relationship between Klopfer's c 


ategories of affective behaviours 


in science education and Krathwohl's taxonomy of affective objectives. 


Krathwohl's categories Character- 
;, |ization by 
Receiving |Responding| Valuing 1 a value 

complex 
o 
€ *- ^o 
4$ % 
CN 
V % 

[3] 


Klopfer's categories 


Manifestation of favourable attitudes 


to science and scientists 


Acceptance of scientific enquiry 
aS a way of thought 


Adoption of scientific attitudes 


Enjoyment of science learning 
experiences 


Development of interest in Science 
and science-related activities 


Development of interest in 


pursuing 
scientific career 


Relationships among various types of 


statements in science 


Recognition of philosophical limitations 


and influence of Scientific enquiry 


Historical perspective: 
the background of Science 


Realization of the relationshi 
Science, technology and economics 
Awareness of the social and moral 
implications of scientific e 
its results 


recognition of 


PS among 


nquiry and 


of Klopfert 


a 


S Science categories.) 


Case Study B: Evaluation of attitude objectives 


Statements! made by the authors of CP7 and providing expansion and clarification of the objectives 
were used by the 'experts' in their judgements. The items were then piloted and relationships 
between pupils! scale scores and other variables (e.g. age and sex) were hypothesized and tested, 
resulting in the establishment of some degree of construct validity alongside content validity for 


the scales. 

The five scales were then administered to 2,815 thirteen-year-old pupils. The responses were 
factor analysed which, speaking approximately, clusters together those items that pupils tend to 
answer in the same way, i.e. by 'agreeing! Or 'disagreeing! with them. If the attitudes displayed 
by pupils! responses correspond exactly to the objectives, then we would expect each 'cluster' of 


items to correspond exactly to each !scale'. 
The results suggested that all the objectives except 


Science) correspond closely to distinct pupil attitudes. 
items for this scale formed an 'Alienation from Science! factor. However, only 36 per cent of 
the variation in responses was accounted for by these factors, suggesting that either the pupils' 
responses were predominantly idiosyncratic, or there are further attitude dimensions that are 
untapped by substantial numbers of items from these scales. 


A3 (social and economic implications of 
A3 had no clear referent, but the negative 


The empirical study 
s, hypotheses and design were 


the research question: 
empirical evidence from 


framework, i 
chers and inspectors, 


ses identified by tea 
lot work. 


1a the absence of a theoretical 
ae to questions and hypothese 
er research and preliminary P! 
,,, This was unsatisfactory for several reasons. 
information! supplied by teachers and inspectors api 
Small studies of attitudes to science have been gumer de 
Unsuccessful in establishing relationships between atti p ag cana 
Relationships appearing in pilot studies are notorious 10 Pi 


A ifferen 

Hypotheses formulated on such pases are tentative and very differe 
ne based on theory. westions were formulated. What are the differences in attitude 
hree categories of research T following integrated science and (b) following various courses, 


TX 
chievement (i) among schools (3) ? jii) among pupils within teaching groups? 
(iii location, denomination and type 


f 'important' issues and 

peared idiosyncratic and often conflicting. 

rous but largely concerned with 'interest! and 
s and other social and personal variables. 
ng in larger-scale work! . 

t in nature from scientific 


The selection Oo 


(ii) among pupils within schools and Sees P ratified by size, 
A sample of 2,815 pupils from ror : een-schools' study. Twelve of these schools formed the 
agi s from these twelve schools formed the 


for the 


of i 
Science course was taken . 
nirty-eigh 


"with; 
aupra Doha sample and t 
ithin-teaching - 1 le. 
ng-group' samp , —T- 
(t Attitude scales relating tO the five s par 
Welve-year-olds) and at the 


t teaching group: 


re administered on entry to secondary school 


Be 

tween-schools study :mde scores for boys and girls separately from each 
ve mean auis ttitudes that could be accounted for by differences 

jation in Sar ial class, IQ, divergency, attitudes on entry 

(see , the science course (integrated or separate 

x grouping, time allocation). 

scores On each scale at the end of second 

ssessed on entry to school. School 

jog OF piP ils' attitudes except A2 (16 per cent additional 


"2 teristi 
charac ü ct on pupi : 4 ; x 
can! tside the cities being more aware of the relationship of 
ou 


i 
the var 
of pupils 


A 

s regression analysis On the f 
md was used to determine 
n characteristics of the group 


e: 

sm was accounted for by à 
vanracteristics showed no signifi 
ariance accounted for, small schoo's 


apactor Analysis and Cluster Analysis: Their Value and 
d Social Review, vol. 2, no. 3, 1971, p. 367-92. 


axter. 3 
Economic an 


1 

J.Raven, J. Ritchie and D- B ht 

Stability in Social Survey Researe™ " 
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Scores only (6 per cent, mixed sex, but nee 
ixed ability, groups with a larger number of periods of science displaying higher 'objectivity!). 
mixi , 


All the scales except A4 (‘interest and enjoyment!) showed Significant increases in scores over 
the two-year period for all groups. These correspond to objectives with substantial cognitive 
OPG na A4 which is essentially affective Showed a decrease for the whole population. 


Within-schools study 


ionships between individual pupil' ^ 
htt pen ulia. gie analysis and patterns of relationships common to Schools following the 
integrated course were searched for. 

No such patterns were found fo 
were combined into one regression. 
characteristics were simila: 
cf. 44 to 62 per cent). 


r this or any other 
The patterns of 
r to those of the 'between-s 


group and the results of all twelve schools 
variance accounted for by pupil xi 
chools' but less Substantial (16 to 29 per cent, 


Within-teaching-groups study 


Three relationships considered were those between a pupil's attitude score at the end of the : 
Second year and school Science achievement, teacher's rating of ‘academic ability', and teacher's 
rating of ‘interest in Science', 

Median correlati 


ted that pupils! 
expectations of them and that the 
aviour is in accordance with the expectation. ) 
Teachers! ratings of Pupils! 'interest! 
(r = 0.29 to 0.39), but correlated 0.82 wi 
teachers based their assessment of pupils' 


were even less Strongly related to pupils! attitudes 


th pupils! achievement Scores. This suggests that the 
interest largely on academic performance. This - 
flecting levels of interest suggests that the teachers m 


that 'schools that are large, 


owing) integrated courses, 
ise procedure, 


in inner-city areas, 

result in unfavourable 
These factors per se exert little 

: E ducation system's control, Y 

i.e. characteristics that g though the effect of the latter is 

much reduced if individual's Scores are considered), 

If we seek educational 
in attitudes, j 


attitudes to Science! , 
influence on attitudes j 


nt (using available psychological 
r-pupil communication, teachers 
classroom tactics and strategies. 
tw.v. Beez, "Influence 
Performance" , in A, M 


Middlesex, Penguin Books, 1968. 


164 


Case Study B: Evaluation of attitude objectives 


ANNOTATED BIBLIOGRAPHY 


Brown, S.A., Affective Objectives in an Integrated Curriculum, Ph.D. thesis, University of Stirling, 
1975. 
This provides a full account of the evaluation including a detailed examination of the 
information available from social psychology about the ways in which attitude goals may be 
achieved. 

Klopfer, L.E., "Evaluation of Learning in Science", in Bloom, B.S., Hastings, J.T. and 
Madaus, G.F., Handbook on Formative and Summative Evaluation of Student Learning, 
p. 559-641, London, McGraw Hill, 1971. 


A classification scheme for affective behaviours exhibited by science pupils. Emphasizes 
attitudes that relate to science as a system of enquiry and carefully avoids ambiguity by 


clear delineation of behaviours. 
Krathwohl, D.R., Bloom, B.S. and Masia, B.B., Taxonomy of Educational Objectives Handbook II: 
Affective Domain, London, Longmans, 1964. 


s that has greatly increased precision 


A hei i ification of affective objective 
heirarchical classifi cpi cy Sr dire ee i 


in this area and improved communication among cur 


Oppenheim, A.N., Questionnaire Design and Attitude Measurement, London, Heinemann, 1966. 


A comprehensive review of, and introduction to, attitude measurement. 
Willis, G.H., "Curriculum Criticism and Literary Criticism", Journal of Curriculum Studies, 
vol. 7, no. 1, 1975, P: 3-17. 

i t of curriculum enquiry that is concerned 
: — is seen as an aesthetic aspec : y Enean 
Ta c acere does this curriculum represent those ideas with which it is 


concerned?'. 


165 


14 Evaluation of the integrated 
science project in Brazil 


Myriam Krasilchik 
Sao Paulo, Brazil 


SUMMARY 


» points and problems faced by a project director 
udy. Expected and unexpected results are outlined 
ems in similar situations. Evaluation of this 
Preliminary rather than final in nature. 


ears which a 
versity. 


taught at Fundamental school a 


S a single subject and at High school 
as three separate subjects; physics, 


: Though general science is part of 
today. 


topies such as living things, matter 
physics, chemistry or biology. 


O de Ensino de 1o grau - Analise descritiva da; 
das Unidades Federadas - ici i 
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urriculum project as involving "group effort to produce some new 
f preliminary materials and feedback from such tryouts 
lum, prior to its release for general distribution", 


i Grobman's ! concept of a c 
hag of approach using experimental tryouts o 
used for the improvement of the curricu 


1s not common in Brazil. 
" Innovation by the science teaching movement followed the Sputnik era. Since 1954 a group of 
razilian educators and scientists? have worked constantly to improve science teaching. They 
later included adaptations of many 


ipt. with the preparation of original teaching material and 
urricula developed during the sixties in the United States and the United Kingdom. In 1964, the 


tintteais È ; 
Initiation to Science" project prepared by the Brazilian Education Science and Culture Institute 
(IBECC) had several characteristics of integrated science. However, the work was the object of 


neither formal evaluation nor of regular revisions. . : 
s In 1965, the Ministry of Education assisted the improvement of science teaching by establishing 
cience Teacher Training Centres in the six biggest B Gereral different groupe! Mm 
^s fferent regions were formed and they worked on the pre tenobing; materiels endis 
evelopment of teacher-training programmes: , . 
Concurrently, a new educational research field evolved on the international scene, namely, 
curriculum preparation and evaluation, with the consequent accumulation of instruments, experience 
and data about the process of curriculum development. However, because of a shortage of 
financial and. even more so, human resources, the Brazilian groups did not utilize the knowledge 
and methodolo ee , ng effort to improve curriculum development and evaluation came 
from the fed e leveloped. us organization of the "Science Teaching Project" which sponsored the 
eral government's B science teaching centres, 


Preparation of a series of projects in different institutions (e.g. 
Universiti at a 
Bien id someone instituten) tion, one may identify several dilemmas and gaps; 
In the present phi f Brazilian education, : À 
T PR e — p = aes for its own cultural expression and a marked exterior cultural 
Bi eni vemen! se "e age need for change and an instability of values; an elitist 
ëducation ex p influence; Pros population which are economic, sona and t pena outcasts. 
sention and lange groups 9f e EeP uono ose Stan aun population Wale 
Tesearch which TEE A answering the demand of the P dace 
i N i alc ` 
Simultaneously presenting genuinely nation? tT consequence for the grou 
ET The need for curriculum evaluation um ted science. 
r penis development, particularly thos The measurement of student achievement is 
Owever tacles Were Therefore, it was necessary to 
the most co — e^ of evaluation among i as warranted in that quality control 5 
dem mmon concep js tment in evaluation w 
Onstrate that the financial investm t of education's potential. It was also necessary to 


analysis would lead to a greater developme valuation of instruction materials would lead to greater 
ent in eva. could really attain the proposed aims. The 


Justi á 

: tency ee in e tenchi nr m pcr ubject and the need for evaluation because 
Med metis neat et ae tr 
evaluation conte amounted 10 502,07 or solve al ipê problems Tbe Pf equation seseareh 
in finanolal roa Beal still vw EY d ua work refers to curriculum 

n Brazila fsom 1968 fo 1978 51994 dd t frequently used technique (48.8 per cent) 


research. In the work carrie out 
Peacock, 1970, p. 4- 


1 projects: 
H. Grob urriculum 
. n ntal C 
man, Developme jon in prazil (mimeographed) , 1970. 


2 


razilian cities. 
paration of 


s a natura ps involved in 
of integrà 


administrators . 


n! 


the mos 


Pibltoation obtainable fro e ( 
Splanada dos Ministerios; Bras! 
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i istribution 
followed by the interview. The most common treatment is percentage ges fa 
es E ds stop analysis, the use of scales and multivariate analysis are rare. In E hres 
analysis. EN ^ teaching programme research, hand calculators are more frequently E 
e en The Offices of Education seldom worry about teaching evaluation due to the n 
cognputers. P ja Raols 
ainly political, which they must solve. " . E 
" n rend mis to develop curriculum evaluation by recruiting social posee iex 
3 i initi eams. 
i to Brazil to help the initia 
tart the work. Foreign consultants came : 
ure ae ei os The Ford Foundation, the Organization of American States (OAS) ano Unesco 
m "udin institutions to systematize and rationalize the field of curriculum projects 
e 
tion and evaluation. EM "m 
eos there are other practical problems: access to computer centres is difficult, veis es 
prolema in instrument application and the communication system is inefficient in some parts 
Ye country. 
à Bert groups still work very much in isolation. Meetings of educational = to 
discuss the results of their work are rare, as is the publication of specialized journals i m 
periodicals which would allow for an exchange of ideas and a critical analysis of work, thereby 
improving quality. 


Against the above background, this 


paper will describe problems faced by a project director 
organizing an evaluation project within 


the limits of local conditions. 
Evaluation of the integrated science project 


Some high schools offer only one 
have a course on one Scientific s 
available. These courses em 
Social aspects of science. 


Science course, with three classes a week. Students normally 
ubject (physics, chemistry or biolo: 
phasize content, are b 
The schools are poor ai 


new methodology suitable to the project bi 
ment of science. 


customed to more c 


Social implications of the develop: 
the part of teachers who were ac 
Production team for the "developmental project" 
i.e. biologists, physicists, chemists, geographers and anthropologists , pro 
that was extremely stimulating, although not always eas 

planning team presented some difficulties in terms of ¢ 
combined with precise allocation of tasks. The inclus 
from the beginning of the work, but administrative con 
them regularly. 


to the project on 

In addition, the 

th differing backgrounds, 
ducing group interaction 
Y. The organization of a curriculum 
reating an harmonious, democratic climate 4 
ion of evaluators in the team was determine 
straints made it impossible to incorporate 


onventional courses. 
was composed of people wi 


Development phases of the curriculum project 


There were four phases in Setting up the project: pre 
financing (1972); project planning, 


-planning, i.e. preparing a proposal for 
to specify goals an 
development by th j 


d aims and to select the contents for 
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Budget distribution by the beginning of the school year had a pronounced influence on the 


organization of the phases. 
" The schedule for producing the experimental 

at the project would end in November 1974. 
States so evaluation suffered as a result of th 


and the evaluation project at the same time. 
In preparing an evaluation project, it appears important to allow a gap of at least six months 


between the end of the developmental phase and the beginning of experimental application. 


edition of the integrated science materials assumed 
The school year begins in March in most Brazilian 
e need to prepare the experimental edition of the book 


Preparation of an evaluation project 


ves negotiations and constant communication to acquaint the 


nancing agencies and the evaluating agencies. From 


nt of mutual confidence and harmony at this stage is 


project. 
What are the main questions that should be 


The first stage of evaluation invol 
institution that prepares the material, the fi 
the institutional point of view, the achieveme 
essential in order to assure a good development of the 


The project director faces a series of decisions: 
asked? What is the best "design" for the study to help in the revision of the project? What is 


the adequate composition of the sample? How should the evaluation team be organized? What 
are the competencies needed? What should be the relationship between the evaluation group and 
the development team? How should the study be conducted, and what are the kind of "unexpected 


facts" that should be "expected"? i 
If we consider that the integrated science project has : 
include several regions of the country, 8 difficult problem given Gene ee a 

account all the differences found among places, it would be difficult and very expensive to try to 
n was solved by using the Science Teaching Centres 


Select a re le. This situatio 
sees an a ee RIA in charge of the supervision of the regional evaluation. According 
8; y six states would have been too expensive. It was therefore 


to the financi i 
a aluation in Y 
eque te pé ap 5 ara in the Sao Paulo, Recife and Belo Horizonte centres only. 
ius y on e analyse the effects of both the materials and the teacher-training 
er decision was ] resources, the training required 


he ili tional system lacks financia 
to gramme. As the Brazilian educa pel na sine ep ss, the training reauired 
guarantee efficiency of the project wo 
at the project co 


One of th eee uld be well-managed with good results even 
e i e 
working hypothes pecific training courses. 


by te 

ea who had not ves whether or not to include a group which would not use the 

maten e was also the questi an s the experimental groups. Since it would not 
rials but which would take the S this nature was included in the project 


affect either expenses Or the wo 
design, 
The main questions to be answered by the 9" 


a national scope, evaluation should 


rkload by much, a group of 


aluation were: 


who had trained teachers and those who had 


1 
1. Are there detectable differences between pupils 


non-trained teachers? attitudes of students and teachers towards science? 
i e 


e results obtained? 


Does the project cause 
n the different levels of the 


3. What is the relation betwee 


4. i 
What content does the pupil 1 

Cognitive domain? 

What should be changed in the project? 


wha 


arn? t are the results il 
e 


e and sex of pupils; time of classes, 


data: 48! 
ting the first year of High school; state, 


" x in cO. 
Six variables were considered » damental sch 
ar and use or 


day or 
evening; grade, final ye 3 
Place of use; teachers! previous pipi 
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Instruments 


A_test based on the objectives pro 


The questions were 
related to the objectives in the co, 


- Thus, the test used 


Attitude scale aimed at measurin, 


g the influence of the 
towards science and implications 


for social problems. 


Questionnaire to check pupil interest and difficulties - consisting of a series of questions with 
open answers. 


Reports 


Project on pupils! and teachers! attitudes 


prepared by the teachers showing the difficulties they found and suggestions for changes. 
Organization 


In January 1975, a summer course was organized for Sixty teachers from the state of Sao Paulo 
and ten teachers from oth, 
teachers, who would beco 


During this period the evaluation 
; the organization of evaluation 

the experimental team and communication 
between the evaluation Sub-groups and the head office. 


ing teachers to use 


t which to use the project iit E. 
( ginning of High School), isi as ma 
include some pre-high school classes, = : 4 ia a fhe decision w 


ted 
i tudy star 
with 926 pupils a hown in Table 25. The study 


Table 25, Integrated Science project design. 


Instrument Application Sample — 
Test before and after 


all pupils 
Attitude scale before and after Pupils and teachers 
Interest i 
est questionnaire after Pupils who used the project 
Report after 


teachers who used the project 


Lats et al, 


Taxonomy OE vss Op. cit. 
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Recruitment of teachers 


Despite the initial enthusiasm, it was very difficult to find teachers willing to use the project. 
From the start, there was noticeable resistance. As the curriculum does not aim at preparing 
the students for university entrance examinations - very important in Brazilian educational life and 
a strong influence in the high school curricula - teachers and students were opposed to the project. 
Some private schools, for instance, refused to co-operate in the evaluation for they depend on the 
influence of the students and cannot risk giving a course that does not meet university entry 
requirements. 

At one evaluation centre, the local supervisors found it necessary to use the force of the state 
Official system to impose the project materials. This led to some disagreeable situations and 


unwillingness on the part of the teachers. 
Ys bans that the participation in a project must be voluntary, not compulsory. Although 


this seems obvious, it may be a useful reminder far those who are going to start a project in 
centralized government schemes. b . y 
BAR conie of resistance derives from teachers who feel insecure teaching a new project. 
In the specific case of integrated science, combining as it does contents, emphases and methodologies 
pop In these cases, support groups formed by the 


fro i h a factor is very important, s 
see dian c the teachers overcome problems. 


e i i are useful to help x 
E o Mi em dee as an institution, there were several different problems among the teachers 
f i ie " jh The introduction of integrated science created competition and worries about 
9 scientific subjects. he other hand, pedagogues and educators who face science teachers involved 
job possibilities. On t eniin may see it as an invasion of their areas and start to defend 
ad. xo of - vs cases, this can be advantageous, resulting in interaction between 
ir territories. , 
Specialists and D cae rie not refer to integrated science, another obstacle to the adoption 
of Ne cc p cpueheo and administrators prefer to remain with the traditional school 
e project. any : I allows them. 
: A which the law : " : 
Subjects rather than enjoy lcge demands only a minimum of very inexpensive equipment, 
Although the integrated scien s to the laboratory material in different regions may 


es 
the difference between the budget gae ria points. 
also represent source of pode their available time and their prestige in the community, 
rsojriscnpo bei cin do tion. Whenever possible, the project director VE headquarter 
are also factors which influence eg ation should agree in the selection of supervisors. The 


i alu à F ; 
Staff and the regional head office of ev h as a science centre or à university which has . 
institution suc -ordinated work environment 


regional office may be an À > central group and a co 
administrative support. Close hos saat Le also have decisive consequences for later phases 


are fundamental to the study 
9f the project. 


tg success. 
were taken to maintain this atmosphere; 
n i oject, several steps t 2 f 
In mie dntegtaien ne eT the regional groups and a general TOENE oe EL era 
central group personnel WO isid These steps Were costly; — E 3 vac ru ie 
€ all supervisors pem chis and group morale and thus they sho included in 
oth in terms of informati 
i f other 
evaluation's budget. information that cannot be obtained by mears ol : 
Teacher meetings are 8 Leer cdm of open discussion is difficult but essential if the meetings 
instruments. To create an pom ccasional comments provide eau - in the reformulation 
i i . s to the other res a. 
are to fulfil their purpose they must be noted and gadeg eec call, fe bieiin —— 
Of the project; therefore t jon and analysis of data is no Ps var cepto 
Although the final compilati diate that despite the re aa Kier B 
i orts 1 i ns allocated for eac 
^": ea La aud ia time AP devoted to each chapter in both the 
ctions occur in 


ing states; the average time 

i boIwORE i f hanges are ve 
i DT fferences i hers; suggestions for change ry 
en eg em a o eeu denm : aiming at formative evaluation will be 
group with trained teac 


nd that w that later projects i 

Similar in the three states and it is ac Me expense while providing an equally accurate evaluation; 
Carried out in fewer regions: 
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i i as they are memorizing contents and gathering 

hes e os oe uname dodi die felony remains that of acquiring factual knowledge; 
ma US i my illustrations which are not referred to by the text; the teachers think that the 
ov ipse =e a change in the methodology of the classes they give. As the pupil text was 
posg SEIA ER ay to induce a specific class organization, including team discussions, independent 
E en P b erben laboratory and simulations, the teachers follow the instructions and 
n dre roles. Véry few teachers resented receiving prepared material which could repress 
bet creativity and freedom to work. 


Were the right questions asked? We may suppose that revisions based on the results of the 
evaluation will make the project more adequate and more interesting to the students. iai 
learn that the training causes differences in teacher behaviour. However, because it is diffic 


to control regional training, the analysis should consider the qualitative variations which occur. 
We may therefore suppose that the courses give greater confidence to the teachers working with 
the project. 


The role of the teachers! guide, an important part in the project, could have been better 
observed. Nevertheless, its effects will be compared to the results achieved by non-trained 
teachers. 


Has the project affected teacher/pupil relations? Although it would be useful to observe and 
analyse these, personnel and financial limitations do not permit it. Direct information from 
teacher reports Suggests that there are changes in teacher behaviour. 

How are the instruments working? In the pre-test Stage, some test questions presented 
problems. There had been no time to analyse and hence evaluate the questions before testing. 
As a result, some questions will be eliminated from the analysis leading to the final evaluation. 
Although initially it was not planned to use tests 


to grade students, they were utilized for several 
reasons, including the fact that students are motivated by grades. 
As the questions on the their analysis represents a 
tremendous task. Though the 


be useful for review, the coding 


evaluation studies. 
In some cases, the 


It has created a more innovative enviro It has 
j ent. 
brought science teachers and evaluators = 


d teachers, 


It has transmitted to teachers 
T à curriculum development project, in th 
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in Malaysia 


Sim Wong Kooi 
University of Malaya, Malaysia 


SUMMARY 


The evaluator has problems enough w 
trying at one and the same time to up 
shift the emphasis from didactic teachi 


are compounded when national policy is seeking t 
Science to society and is requiring change in the language of instruction from English to the 


National Language. This chapter illustrates the problems of evaluation against a complex 


background of pedagogical reform and of changes in society. 


hen he is attempting to assess a course in science which is 
-date its content, to introduce an integrated approach, and to 
ng to learning through active discovery. But his problems 
o generate an awareness of the importance of 


Prologue: a Malaysian scenario 

It would not be incorrect to Say that Malaysia has been pursuing two virtually antithetical trends. 

On the one hand, it is trying to keep pace with more advanced countries by embarking upon Science 

and mathematics curriculum reforms on several fronts simultaneously. i One avowed objective has 

therefore been the "improvement of the quality of — for the building of a progressive society 

Oriented towards modern science and technology". On the other hand, in keeping with the trend 

in newly-independent countries, which strive to establish national eom pomis has also 

invested in the progressive implementation of its national language ape mes aysia) policy. 

By 1975, the medium of instruction for all subjects in the former English me um primary schools 
aysia. 


had been converted to Bahasa Mal This has posed problems because the bulk of the 
teachers are non 


qualif hematics -Malays who lack either basic proficiency in 
ified science and mat! ema hing in Bahasa Malaysia, or both, and the national language itself 
o! in tence ted (albeit rather confused) process of coining 

ence an 


i iculum reforms 
ientifi f the many sci d mathematics curric 
scientific terminologies. — T5 mie urged as early as 1968 that "we should not 


kia ware senses Dei could wait. : is ip spe! bo o the whole process 
of curriculum development and it ought to be thought of rig ae serene bicelles Te 
: í ful that the authorities concerne y i ols have 
While the writer is hope uld embark upon a full-scale summative evaluation of integrated 
adopted the new curriculum - oes a kind of reflective evaluation in order to consider yet further 
Science which might even poc? initial efforts might be illuminating. In subsequent sections, 
s 


Science curriculum reforms, 


1 Lumpur, Government Press, 1971, p. 232. 
Second Malaysia Plan, 1971-1975, Kuala p 

"Processes of Curriculum Innovation in Science Education" , 
p. 298-311. 


on: 
bigs 1969, vol. 2, no. 1, 


2 
Sim Wong Kooi. In 4 a 
(3lst July 1968), Masaalah Pendidikan, 
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apart from indicating desirable directions for the comprehensive, continuous and co-ordinated 
Aes of integrated science, a few evaluation projects which have included some aspects of 
integrated science will be reviewed, even though they fall short of the total expectation. 


Overview of implementation efforts 


The integrated science curriculum in Malaysia has its origins in Scotland, where the syllabus was 
meant for a two-year course. The syllabus was modified to meet local needs and additional 
materials were incorporated for the three years of lower secondary school. Beyond the : 
Similarity in name and in the adopted curriculum materials, there is very little resemblance in 
the context and manner of implementation. In the first place, a prevailing preoccupation with 
examinations seems to permeate pedagogical practices in Malaysian schools. ^ Results favour 
teachers who provide copious notes and drill their Pupils on past examination questions. Attempts 
have been made to incorporate questions whic 


h go beyond "knowledge" or which relate to practical 
activities. However, not only are these types of questions no longer drill-proof, but, with the 
time and personnel constraints in Setting examination questions based on a rigid multiple choice 
format, the proportion of such questions is by no means high. Scottish teachers, on the other 
hand, are not required to prepare their Pupils for a public examination at the end of the two-year 
integrated science course. Hence, they can provide a less deta: 
occasional departures from) the topic 
than their Malaysian counterparts, 
in case the examination questions s 
even when the two 


take initiatives for which they m 
unquestioned acceptance of autho 
objective-type examinations, wo opment of inquiry skills in pupils. 
In contrast, Scottish teachers have had a long tradition of e i 

what curricula they wish to ado 


pt. Thus, unlike Malaysia, where all schools will eventually be 
required to adopt the integrated science Syllabus and "trial" 


low directives passively rather than 
ay be taken to task. It is 


On top of this, 
"equipped, and typically, teachers 


Hopefully, with 
ll have less cause to 

1 

S.K. Pang, "Integrated Science in a Mala 


ysian Context" ga; ; 
Selangor Science Education Association), ; S2ins dan Hisab 


the 
1975, p. 20-26, — ——31525 (a Journal of 
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complain. While the situation may be said to be improving, it is still a far cry from that of 
Scotland. 

Having pointed out some of the salient differences between the context of integrated science in 
Malaysian schools and its progenitor, it might be useful to examine more closely its nature and 
mode of implementation. First and-foremost, it is important to examine what is meant by 
"integrated? . Whatever the approach, the critical emphasis is on the fundamental unity of science 
which cuts across traditional subject boundaries. Integrated science in Malaysia, which might be 
regarded as a mixture of conceptual and environmental approaches, places emphasis on three broad 
aspects, namely, "developing an understanding of environmental phenomena", "developing a 
functional understanding of the inquiry skills and modes of thought of science", and "developing a 
healthy attitude toward the application of scientific knowledge". 1 Scrutiny of the sixteen syllabus 
Sections, however, does not indicate a degree of thematic integration much beyond that which 
obtained with the former general science syllabus, as shown by the distribution of the topics in 


Table 26. 


Table 26. Distribution of integrated science topics by subject emphases. 


Equal emphasis on 
Predominantly 1 subject 2/3 subjects 


Basically 1 subject l 
: 2 Biology, 


" : à š à ' Biology and 4 
Biology Chemistry Physics : Biology Chemistry Physics RENIE Chemistry 
: : and Physics 


12. Transport 4. Particles 3. Energy 15. Support 1. Introducing 


2.Looking —9.H 1. Methods 
9. Hydrogen f tt : and Science 
at living acids and of heat : systems ofimatter 10. Detecting movement š 
things bases transfer — ; 5.Some the environ-: 
: t 
6. 8. About common men 
Cells and 11.Solutes CAHEE 


reproduction and Solvents electricity 
13. More 16. The earth 
about 


electricity 


14. Fluid 
pressure 


the first year of secondary school (Form 1) while sections 7-11 and 
E II) and third (Form III) years, respectively. The scheme 
d ears py Figure 15 Although the target date for completion 
a few schools had still not been able to convert to 
d teachers and equipment and the absence 
for it must be 


Sections 1-6 are meant fo 
12-16 are meant for the secon 
for progressive implementation is dep 4 
of the implementation to all schools Ww pw 
integrated science, mainly because of the spud 
of laboratories. | Repercussions wdygc- "m aper 
Very uneconomical to continue to set a separa P: 


of traine 
lized examination system are likely 


for these few schools. 


aysian Schools Form I-II (Provisional 


i Mal 
E Pelajaran. Inte rated Science for 


Syllabus), May 1975, P- 1- 
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Figure 15. Progressive implementation scheme for integrated science. 


Year: 1269 1970 1971 1972 1973 1974 1975 
Form III 22 62 153 174 

Form II 

FormI 

Total 


For three successive years, the first batch of inte 
Schools attended in-service workshops conducted by a 


practice is for the teachers to go through the worksheets, together with the accompanying activities, 
and to discuss problems encountered. i 


limited to vacations. 


eptions regarding problems, 


appealing, with a lively 
teacher morale. 


: piis the — segen of the implementation of integrated science enthusiasm was high, 
not only on the parts of the officers-in-char e but also a à etime 
qued tap o g mong the teachers. In contrast, som 


ate of newspaper article 
which suggested that all was not well. The report of a Saloos Heads' 
stated; "Heads of Secondary Schools toq: i - 


A resolution adopted the closing day of the twelfi s 
th of 
Secondary Schools read: "The Suspension is ene conference of neatly 500 heads 
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may become innocent victims of the poor pilot scheme and end up as dropouts". 1 

This situation prompted the Ministry of Education to respond by convening a workshop to 
examine the validity of these statements. Since the bulk of those who attended comprised mainly 
Ministry officials and teachers who have been recognized for their enthusiasm, it comes as no 
surprise that after considering a series of point-by-point rebuttals, the inevitable conclusion was 
reached that the furore was but a storm in the tea cup. 

It should have been evident that what was urgently needed would have been a well-planned and 
well-executed summative evaluation of integrated science. While there are plans for such an 
evaluation, there has apparently been some reluctance to exploit several possible evaluation avenues, 
svch as careful re-analysis of data collected from periodic feedback questionnaires, scrutiny of 
Several past reports involving the evaluation of some very specific aspects of integrated science and 
other modern curricula, and capitalizing upon the willingness of some researchers, who have 
perforce been required to conduct evaluation studies on a reduced scale. 

As far as the writer is aware, only one attempt has been made during the initial stages to 


summarize the potentially useful source of teacher feedback, although more systematic analysis 

might have provided useful insights for more relevant decisions. As the number of teachers 

increased in subsequent years, even rather simplistic analysis was not undertaken, partly because 
s task and the officers-in-charge were not able to derive more 


it became an increasingly onerou 4 à 
meaning from the data other than certain interesting open-ended remarks which were commented 
upon in the newsletters Perhaps the assistance of researchers might have been enlisted not 


only in the data analysis but also in improving the questionnaire format to facilitate processing by 


computer. 


Evaluation vis-a-vis equipment aid 
ware of the need for schools to be better equipped for teaching 


i i integrated science. Accordingly, in the first set of United 
irent imt s ren b modios to the Ministry, ipee d por Pm eg formed a 
major component and the list of equipment sought was geared to the di F o| etu S WR 
teaching. Prior to the next phase of Unicef aid to Malaysia, an evaluation 
programmes was undertaken. indi ted here. They arise from interviews with 
i nt findings are presen . 
dence a dod di pipi of science equipment, — w wit ote um 
questionnaires to a sample of schools ep ch or Ed integrated science in 
and field interviews with principals and teachers who teach or supe g 
schools, ? i led so as to represent three 
" P i Unicef, forty were sample i 

TG na 125 ee re c kies general accessories, such as P M s 7 

gories of scien teen items), specialized accessories, such as platinum electro! les, 
thermometers: anā beskars (E it items), and specialized units, such as energy conversion 
hydrometers, and microscopes (thirteea ; In order to make more a meaningful 


fourteen items). à 
kit, human eye model and pH noy pte were suggested, showing congruence, in terms of 
e 


interpretation of the data, a Ix SOA to whether or not the aid was received (RA) with 

sa e peg in ia gente has the item (AH) or whether or not the item is frequently used 
whether or not the scho 

(FU), as shown in Table 27. 


The Ministry of Education is fully 2 


lsStraits “imes, 9th July 1973. 


i i Og: he Ministry of Education 
f Unicef Aid Programmes to t f . 
"sim Wong Kooi et al, An Evaluation B, (A condensed version also appears in Jernal Bahagian 


FERMEN n Pelajaran Malaysia, 1974, p. 87-139.) 
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Table 27. Indices of congruence/incongruence. 


Type of (1) Whether (2a) Whether (2b) Whether 
congruence/ received School already frequently 
incongruence aid (RA has aid (AH used 

C+ Majority Yes Majority No Majority often 
Ga Majority No Majority Yes Majority not often 
I+ Majority Yes Majority Yes Majority not often 


T= Majority No Majority No Majority often 


——— HM —À—MMM—————M €—————— € E 
Alte-natively, the indices may also be summarized in the form of the matrix shown in Figure 16. 
Figure 16. Indices summarized in the form of a matrix. 


AH (or FU) 


No (or often) Yes (or not often) 


RA Yes (or not often) 


No (or often) 


When the items were grouped according to the types pf congruence/incongruence as well as the 
three categories of equipment, it was possible to identify items which displayed the I- type of 
incongruence and hence requiring urgent attention. 

When the responses were compared according to various Sub-groups, however, some 
characteristic patterns became apparent, as shown in Tables 28, 29 and 30 j 


Table 28. Distribution of congruence /incongruence patterns according to location. 
Comparing RA with AH Comparing RA with FU 
I- I+ C- © I- I+ C- C+ 
Urban 
Rural 
Table 29. 


I> T$ cs œ 


I- F C- c 


Single sex 


Mixed sex 
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Table 30. Distribution of congruence/incongruence patterns according to medium of instructi 
ction. 


Comparing RA with AH Comparing RA with FU 


I. t g- œŒ Is de (e C 

National-type 30 0 6 4 13 i dB 3 
(English or Chinese medium) 

7 9 4 20 4 16 7 13 


National schools 
(Bahasa Malaysia medium) 


Thus, "in terms of incongruence, the urban, single sex and national-type schools tend to have 
proportionately more incongruence of the I- type than of the I* type. Similarly, proportionately 
more of these types of schools have congruence patterns of the C- type than of the C+ type. It 
may therefore be concluded that in respect of these schools the majority do not seem to be 
receiving aid" 1, even when they do not already have equipment or are using the little they have 
to the fullest. On the other hand, for rural, mixed and national schools, aid has been received 
by the majority even though many already have sufficient equipment or they do not use the little 


they have frequently. 

While the questionnaires (which were concerned with orders of priority of items and how they 
were being used) coupled with school visits largely confirm the findings from the analysis of 
proforma returns, some noteworthy aberrations were observed, such as the following - 

"... electrically operated ite o schools in certain rural areas that at present do not 


have electrical supply, althoug d the hope that they expect to have 


electricity in one or two years! time." 
one set of equipment were distributed to different schools who 


"In one state at least, parts of 
quipment until and unless they obtained the other parts." 


therefore could not utilise the e 
which were related directly or indirectly with equipment aid as well as 


Many other aspects, ‘ 
integrated science, were also considered in the report, but they are quite beyond the scope of the 
present paper. It suffices however to emphasize that from the point of view of schools, the 
availability of adequate and relevant equipment contributes a great deal to the likelihood of 
successfully implementing the practical-oriented integrated science curriculum. 


ms were sent t 
h many of the principals expresse 


econd Malaysia Plan (SMP) 
over-riding objective concerns the "improvement of the 
ressive society oriented towards modern science and 


technology" - 3 The Ministry of Education was accordingly interested in its mid-term review of 
the Second Malaysia Plan (SMP) to ascertain the extent to which the various curriculum reforms in 
science and mathematics education are contributing towards the attainment of the SMP goals. A 
team from the University of Malaya was therefore commissioned to carry out such an evaluation. 
Although it is not possible to discuss all the findings , j few interesting results pertaining to 
integrated science will be highlighted. The over-all finding was that a major problem confronting 
the science and mathematics curriculum reform lies in a kind of paternalism which comprises an 
nattitude that has a tendency to stifle initiative and reinforce over-dependence". The over-all 
recommendation was also characteriz which was interpreted as the "maintenance 


Evaluation vis-a-vis the S 


From the standpoint of national planning, 8n 
quality of education for the building of @ Prog’ 


ed by "Janus-face" , 


laim Wong Kooi et al, An Evaluation ..., OP- cit., p. 103. 


_, Op. cit., p. 115-9. 


? Sim Wong Kooi et al, An Evaluation .. 
,p. 232. 


aysia Plan, OP- cit. 
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of an orchestrated perspective, involving having a look in at least two directions vci ege a 1 
These conclusions were discussed in terms of roles in curriculum reform, teacher pra abel - 
demand, problems of Bumiputra pupils, curriculum co-ordination and improvement, am feu. i 
objectives, research and evaluation; Pr MU up in the form of two frontispiece 
i i i "Paternalism" an anus-face". e 
Ep oci anie Mae a series of brainstorming session with various groups of Ministry officials, 
namely inspectors of schools, officers in charge of planning and research anc officers involved in 
public examinations, administering questionnaires to in-service participants and field interviews: 
The methodology employed was essentially one of comparing the various newly-introduced curricula 
in science and mathematics and the main concern has been in terms of SMP goals. 
paper reports only a few of the findings pertaining Specifically to integrated science. 

Among other things, participants at the various in-service courses were asked to name three 


main objectives of the science/mathematics programmes. The responses of the sixty-five 
integrated science participants are summarized in Table 31; objectives given by fewer than five 
respondents have been omitted. 5 


Table 31. 


The present 


Perceived objectives of the integrated science programme. 


eo 


Objectives — € Per cent 
Enhancing interests/motivation 28 43.1 
Improving/changing teaching methods 23 35.4 
Learning to handle/perform practical activities 18 27.7 
Curriculum renewal for national progress 8 12.3 
Improving confidence in teaching 7 10.8 
Improving knowledge of science 6 9.2 
Improving teacher's knowledge of science 5 7.7 
Learning by discovery 5 7.7 
Curriculum renewal for requirement of business and industry 5 7.7 
Improving competence in using aids/apparatus 5 7.7 
Integrating various sciences 2 3.1 


m 
The respondents were concerned firs 


] then with psychomotor objectives. 
Interestingly enough nly two respondents. 


through i toe: t ) ith improvin 
through in-service training) aimed princ; pe sis uam : 
that are in keeping with national progress." 2 . ng pup ith new 


"sim Wong Kooi et el, A Position Paper TY 


Op. cit., p. 38. 
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The teachers were also asked to identify factors which helped or hindered the implementation 
of the new programmes. A characteristic feature, as is also the case with the other new 
curricula, is the existence of corresponding positive and negative factors, suggesting that some 
schools are comparatively well off while other schools are lacking in respect of particular 
Pre-requisites for successful implémentation. A particularly pronounced bipolar response concerns 
the availability (41.5 per cent of responses), or lack (53.8 per cent), of apparatus /aids /facilities. 
An important negative factor is associated with language difficulties (32.3 per cent). Since pupils 
have difficulty expressing themselves in filling in the worksheets and the examination requires them 
merely to select from multiple choice responses, the practice of dictating responses, with or 
without having carried out the practical activities, is unfortunately reinforced. Another important 
negative factor seems to be the heavy teaching load (26.2 per cent). A directive from the Ministry 
to schools suggested that, as far as possible, integrated science teachers should not be given more 
than twenty-five teaching periods per week. However, this did not take into account earlier 
directives which suggested that teachers with specialized functions (such as guidance teachers, 
library teachers and physical education teachers) also should not be given more than twenty-five 
teaching periods per week. Again, this has tended to provide the less diligent and dedicated 
teachers with some justification for cutting down on pupils' practical activities, especially since 
they are time-consuming in terms of preparation and instructional time and may even be negatively 
correlated to performance in the convergent, objective-type paper-and-pencil examination. 

Given what seems to be a rather discouraging environment for practising inquiry, we are 
curious to know what conception teachers had of "discovery". During the field visits, the sample 
of teachers interviewed were, inter alia, asked to define the term "discovery". The various 
responses are summarized in Table 32 by classifying them under certain broad headings. The 
classifications conform roughly to Wittrock's suggested trichotomy in terms of independent, 
dependent and intervening variables. l The independent variables refer to modes of instruction, 
and the "discovery" method is used as a contrast to such pejorative terms as "traditional" , 
“expository, "teacher-centred' and "didactic". ^ The dependent variable refer to the goals of 
Instruction and the achievement of a particular discovery would be one of these goals. The 
intervening variables refer to the internal processes of learners involved in an "act of discovery". 
Processes include integrating the material learned into one's cognitive structure or acquiring a 
Positive concept of self as an autonomous problem solver. The teachers interviewed seem to have 
touched on all three aspects, although with less emphasis on discovery in terms of dependent and 
intervening variables. fe 

Further probes however revealed that the prevailing conceptions tended to be rather superficial 


ila Saute, For example, in the conception of discovery as an activity, the concern has not been 
ana in terms of the mental activity involved but often in terms of the associated physical 
ivity. 


approach When discussing the practicability of conducting lessons which reflect the parce M 
pupil b. » most teachers tended to confine their attention to such questions as: How Los 
n usy with doing something, given the limited resources? How to complete the sy . 
‘thin the limited time available? How to get pupils to work on their own without raising the noise 
level in the classroom? ? 
Important though these questions are, they do not come to grips with the crucial issues, such as 
those Pertaining to the careful differentiation among different conceptions of discovery and the 
delineation of Specific arrangements needed to gnsure that every pupil, as well as the teacher, 


wo A 
uld be able to engage in meaningful inquiry. 


1 
M.C. Wittrock, "The Learning by Discovery Hypothesis", in L.S. Shulman and E.R. Keisler (eds), 
Learning b Discovery: A Critical Appraisal, Chicago, Rand McNally, 1966, p. 33-75. 


Sim Wong Kooi et al, A Position Paper ..., OP. cit., p. 80. 
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Table 32 Conceptions of "Discovery" among sample of teachers interviewed. 


R ee 


Frequency (As a percentage) 
-—————— OOO 
Discovery defined as: 


(3) Activity 


1. Self activity 44 (28.0) 

2. Experimental activity 36 (22.9) 

3. Guided activity 26 (16.5) 
(b) Learning 

1. Learning by doing 16 (10.2) 

2. Incidental learning 4 ( 2.5) 

3. Learning to learn 2 ( 158) 
(c) Internal inquiry 

1. Reasoning/relating 19 (12.5) 

2. Environmental awareness r ( 4.5) 

3. Re-discovery 3 ( 1.9) 


Se eee 


Source: Sim Wong Kooi et al, A Position Paper ..., op. cit., p. 80. 


Some current efforts 


» the data collection for which was completed 
s. Am overlapping cross-sectional longitudinal design has been adopted, 


as shown in Figure 17, so as to enable us to investigate possible changes in attitudes as well. 


Figure 17. Overlapping cross-sectional longitudinal design. 
1975 1976 
Form I 
Form Il Form II 
Form III Form IN 


This permits comparisons of the Sàme group of Students 
groups of students in the same form. 
towards science in general and 


; in particular the 
nce in general and 
Student backgrounds 


dents towards Scie 
the relationships between 


! riy Suk Men Chew, Private Communication, 1975. 
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(in particular their sex, ethnicity, self-concept, family Socio-economic status and family support) 
and their attitudes and attitude changes; the relationships between teacher attitudes, especially 
towards integrated science worksheets, planning, instruction and evaluation, and the inquiry approach, 
their academic and professional backgrounds and their teaching styles, and their students’ attitudes 
and attitude changes; the relationships between student attitudes and attitude changes and academic 
achievement; intercorrelations among student attitudes towards science, inquiry approach and 
worksheets; intercorrelations among student self-concept, family socio-economic status and family 
Support; and intercorrelations among teacher attitudes, their backgrounds and teaching styles. 

The design of the study may therefore be depicted by the diagram shown in Figure 18. 


Figure 18. Study design. 


Independent variables Additional variable 


A. Student achievement 


A. Student backgrounds; 


1. Sex 
- Ethnicity Dependent variables 


2 
3. Self-concept 
4. Family socio- A. Student attitudes toward: 


economic status 1. Science 
- Family support 2. Inquiry approach 
3. Worksheets 


Intervening variables 


A. Teacher attitudes toward 
integrated science: 


1. Worksheets 

2. Planning, instruction 
and evaluation 

3. Inquiry approach 


. Teacher background: 


1. Academic 
2. Professional 


. Teaching style 


Mea, 
Surement of these variables will be made through: 


1. ini 
Administration of a student questionnaire, which comprises the following sections: 


A. Background information. 

B. Family support regarding studies, especially in respect of science. 

c. Self-concept regarding studies, especially in respect of science. 

D Perceptions regarding frequency of various pedagogical practices associated positively or 
negatively with the inquiry approach. 

E. 


Attitudes towards science (science teachers and scientists). 
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2. Administration of a teacher questionnaire, which comprises the following sections: 


A. Teacher background. 


B. Attitudes towards planning, instruction and evaluation of integrated science lessons. 


C. Attitudes towards worksheets, including experiences with each of the sixteen sections. 


D. Attitudes towards inquiry approach. 


Observations of classroom teaching to determine teaching styles of integrated Science teachers. 
It was hoped initially to observe a sample of classes at different times in 1975 in order to 
develop a more systematic observation instrument which might provide contrast situations 
between the type of inquiry! approaches adopted by the more indirect and student-centred 
teachers and those who employ a more direct and teacher-centred approach. This instrument 


would then be used at various times in 1976. especially at times which are proximate or 
distal to school or public examinations. 


Semi-structured interviews of teachers and students. 


While items would be similar to some 
of those included in the attitude items in 1. and 2., the interviews would also concentrate on 


Specific problems encountered and general interests and motivations. 


Student achievement tests, including school and public examinations. 


If possible, sub-scores, 
such as for those items corresponding to various categories of Bloom's taxonomy^ or other 


alternative classifications associated with or with relevance to practical work and everyday 
applications, would also be obtained. 


With the constraints of time, finance and personnel available, only a sample of English-medium 
Schools in four of the eleven states of Peninsular Malaysia would be included, and these would be 


further stratified on the basis of year of implementation (1969, 1970, 1971, 1972 and 1973), sex 
composition (boys, girls and mixed), school size 


(more than 1,500, 1,000-1,500, and fewer than 
1,000) and location (rural and urban). Likewise, classroom observations and interviews with 
teachers and students would be confined to schools in Selangor. 


Epilogue: towards an integrated evaluation strategy 
An over-all finding which seems to be emerging from the variou 
or informal, is that the implementation of inte 
relevant educational as well as social, 
embedded. 


S evaluation efforts, whether formal 
grated science has not been integrated into the 
cultural, political and economic System in which it has been 
Thus, apart from the need to integrate the curriculum horizontally with other curricula 
Which entail approaches diametrically opposed to the so-called inquiry approach, and vertically with 
developments in science curricula at the primary, upper secondary or even the tertiary levels, it 
is imperative to consider seriously social expectations based on examination performance, cultural 
disincentives to question authority, irrevocability of political decisions and the severe limitations of 
recurrent funds in schools for proper implementation of integrated science. 

Likewise, if evaluation is to be meaningful and effective, it should take cognisance of the 
extent to which intended, as well as unintended, objectives h: 


ave been attained, and to mobilize all 


1 
It is envisaged that, of the possible t i i i 
, ypes of discovery lessons (such as the "Op tive", 
® Structured inductive", "Semi deductive® i fiche 
"transductive" (as expli: i 


“Simple deductive" , "hypothetico deductive? and 
Sec, a Pes “4 Harold Morine and Greta Morine, Discovery: A Challenge to 
» ot.» trentice-Hall, 1973), onl i 
others are likely to be rare. à TH TERI wei p 


edominate in our schools while 


2 
Bloom et al, Taxonomy of ..., Op. cit. 
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potential manpower and media for the continuous, comprehensive, concerted and co-ordinated 
evaluation of integrated science. Alternative assessment instruments need to be explored for 
evaluating important affective, higher cognitive and psychomotor objectives of integrated science. 
A recently developed simulated practical chemistry test (using slide presentations) appears to be 
equally if not more effective than the conventional practical test which is costly not only in terms 


of money but also in respect of time, energy and resources. While such a test is important in 
assessing pupils' ability to make observations and inferences, practical manipulative skills can 
only be adequately assessed over a period of time by teachers. Teachers are in fact an important 


Source of manpower for evaluation studies, for they could be encouraged to carry out mini-studies 
based on detailed specifications of alternative projects. With the increasing interest among 
teachers in associations for science and mathematics education in Malaysia, it is conceivable that 
Such associations can play a crucial catalytic role. The Curriculum Development Centre can then 
play the more important role of co-ordinating the various efforts by teachers and researchers, 
instead of having to undertake ad hoc evaluation studies. To standardize the conduct of criterion 
tests, it might be possible to exploit alternative media, such as radio and television, for the 
Simultaneous administration of special tests and questionnaires on a nationwide basis. 

Some fundamental value questions need also to be critically examined vis-a-vis the desirable 
Purposes, processes and products of introducing integrated science. For example, as depicted by 
the accompanying photographs of typical laboratory situations, there seem to be abundant indications 
of pupils busy doing science, but are they really learning science? While the answer may be 
arrived at empirically, the philosophical, socio-cultural and even political concomitants appear to 
be many and varied and should be examined or re-examined in concert. 

Whether the Suggested integrated strategy of evaluation is feasible or even desirable within our 


pam paternalistic system remains an open-ended question, but one that cannot any longer 
e avoided. 


16 Evaluation of integrated science teaching 
in Japan 


Hideo Ohashi 
National Institute for Educational Research, Tokyo, Japan 


SUMMARY 


In this case study, student achievement and performance are evaluated in order to obtain useful 
data for the improvement of the national "Course of Study", curricula and instructional methods. 
The ensuing discussion delineates the effects of the mode of evaluation on Student attitudes toward 
learning science and indicates the necessity for a more comprehensive kind of evaluation. 


Evaluation of the course of study 


The "single-track" 6-3-3 school system was inaugurated in Japan in 1947. Attendance at the 
Six-year elementary school is compulsory from the age of six, as is continuation to the three-year 
lower secondary school for those who have completed elementary school. A child who has 
completed lower secondary school may go on to an up| 

types of upper secondary school courses: 


years or more. In terms of contents 
in two categories: general and specialized. 


se i 
area as their future career. Medis. whid liave chosen a particular vocational 
The basic standard for school curricula wi 
for different subjects are outlined in the natio 
Education for each of the three school levels, 
Council, an advisory body to the Minister, 
Integrated science is taught from the fir 
lower secondary school. The science cours 


hich includes the obj 
nal "Course of Study" 
on the basis of the 


ectives and teaching materials 
Prepared by the Ministry of 
recommendation of the Curriculum 


Survey of the scholastic achievement of students 


The effects of this new school system and the course of stud 

" f ly have been 
points of view. In particular, heated debates were held among WP ois a from od 
Scholastic achievement of students is rising or declining. To Secure data 5 Ban to whether the 
since the early fifties various achievement test progr earing on this problem, 


ammes have been 
municipal boards of education, and by public or private educational niani cian or 
s. 
e 


Since 1956, a nationwide achievement test Survey has been cond 
ucte: 
of Education. The test used for the survey comprises approximately à ee by the Ministry 
thirty sub-questions in all, per subject. The time allotted Ons, totalling 


per subject i i 
the items are in the form of objective, multiple-choice questions R mL. ) minutes. Most of 
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Example. (One of the anchor problems in the science test administered to eighth and ninth grades 
in 1965.) 


The following diagram shows how life under water is inter-related. Study the diagram and answer 
questions 1 and 2 below. 


Figure 19. Life under water. 


X MI 
O S 
S zooplankton e 
PELO 


phytoplankton — -killifish 


L Each of the living organisms in the diagram mainly takes in either A or B in the water and 
breathes out the other during the daytime. The solid arrows show the routes by which they 
are taken in and out. What substances do A and B stand for? 

a. oxygen b. carbon dioxide c. nitrogen 
d. carbohydrates e. fat f. protein 


correct answer; a and b 


facility: 2 26.6 per cent in eighth grade 
27.4 per cent in ninth grade 


The dotted arrows show the food chain between phytoplankton, zooplankton, and the killifish. 
Which of the following has the same sort of relationship? 


a. root nodule bacteria — pea plant — man 
b. worker bee — male bee — queen bee 

9. water-flea -- killifish — mud snail 

d. rice plant — pearl moth — leaf hopper 
9. cabbage + grub — sparrow 

f. 


rose -- ant — plant-louse 
correct answer: € 


facility: 25.5 per cent 


l 
Ministry Of Education, Zenkoku Chigakko Gakuryoku-Chosa Hokokusho, 1965, Report on the 
Nationa] Survey of Scholastic Achievement of Lower Secondary School Students in 1965, 
esearch Section, Ministry of Education, Tokyo, 1967. 


Percentage of students who answered correctly. 
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The results of the test and their detailed analysis were published every year as an annual 
report, and distributed to schools and boards of education. Listed below are some important 
findings accumulated over the past ten years as a result of these surveys. 

First, more than one thousand problems with their respective "facility" percentage on the 
elementary and lower secondary school science have been accumulated. These results are 
invaluable as a basis for designing science curricula and planning instructional methods. 

Next, there are wide variations in the achievement levels of schools. In the case of ninth 
grade, results showed a discrepancy of eighty points for a 100 point test between the highest ranking 
school and the lowest ranking school. The standard deviation of the school scores was shown to 
be around eight points. 

In addition, student achievement not only reflects individual abilities and efforts but it is also 
"largely dependent on family background, organization of teachers at school, the size of the School, 
the availability of facilities, the curricula, urbanization, economic condition and the educational 
concerns of the community. 

There is also a wide range of difference in scholastic achievement, even between schools whose 
pupils are from similar educational backgrounds and economic and social conditions. 

Finally, other points include the fact that a student's achievement on a test is not only 
determined by the difficulty of the questions but is also dependent largely on the instructional 
methods. 

Although this nationwide survey was proven to be an invalua 
criticism has been voiced concerning its administration. 
was conducted on the basis of a paper-and 
can be obtained, there is a tendency amo; 
results. In particular, these surveys a 


S For all these 
in 
by the National Institute for E ducational Hesekeoh, Get on: ce 1975 surveys have been resumed 


small 
schools and with diagnostic test problems on some specific topics entered E 
Survey of teachers! opinions 


In revising the course of study, considerable importance is gi 
Teachers Associations formalize the opinions of Eon e ae 
the public. 


i i opinions 1 i 
information necessary for the improvement of compulsory a ied Aries aey 
g etermine the 


ematie by those concerned 
cessary. 
education, considerable emphasis and interest were focused on 


Sape In terms of compulsory 
individual child's abilities. est means of developing each 


1 National Federation of Educational Research Institu 
Iken Chosa. (Report on the Survey of the Opinio 
Education), National Institute for Educational Rese 


open be Sumukvoticu Kaizen ni kansuru 
ng the Improvement 
arch, Tokyo, 1971. T aer 
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Some fifty questions were drafted dealing with such basic factors for the development of 
children's abilities as the problems of: child development, educational systems including grade 
Systems and class formation, the scope and sequential ordering of instructional materials and the 
diversity of teaching methods and pre-service and in-service teacher training. 

The following is one of the fifty questions: 


The course of study contains outlines of materials to be taught. As a principle, every pupil 
is expected to master the material. Under this system, what percentage of the pupils do 
you think have an over-all understanding of the material? 


1 Over three-quarters of the pupils. 
2. About half of the pupils. 

3. About one-third of the pupils. 

4. Less than one-quarter of the pupils. 
5. Uncertain. 


The questionnaire was distributed to teachers in 727 randomly-selected elementary schools 
and in kindergartens and lower secondary schools in the same school districts as the elementary 
Schools, to teacher consultants on boards of education and to researchers at research institutes. 
Table 33 Shows the distribution of the responses. 


Table 33. The degree of comprehension of the material. (Expressed as a percentage.) 


H 
g È 8 
E 8 3 
8 z E i E 
Tä £23 82 85 8 
$ d Ta Ca zi 
o & oo v Q gg ~= 
2 3a 688 BR ga @ Em 
S, »2 82 388 ag Ek $2 
4 > ics Big 2a 3. 8 "E 
£ 53 <3 xs Ho 35 ga 
. B * . a 
E E e eo + n B E 
Elementary school teachers 1.1 2.2 4.5 
L 
ower Secondary school teachers 1.1 16.7 50.2 26.1 4.1 1.8 1,884 
T 
eacher consultants 1.4 29.9 50.5 10.8 0.7 6.6 | 2,361 
Sta 
ff members of prefectural 1.4 20.1 50.0 12.8 1.5 14.2 | 1,032 


€ducationa] institutes 


S 
taff members of municipal 2.0 20.6 43.8 20.9 3.3 9.5 


| 30 
educational institutes | 


Generally, Speaking, the greatest number of responses was that about half of the pupils 


und 
ee the materials. | Nevertheless, a considerable number of educators responded that less 
e 


ditio -third or one-quarter of the pupils understand the materials. Table 34 shows the results 
Süblear. On the degree of comprehension of lower secondary school pupils for each different 
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Table 34. The degree of comprehension of the material (for courses taught in lower secondary 
schools). (Expressed as a percentage. ) 


- b 
& "d n z uU à u 
j$ 19 31 3 
E 5a 68 §8 gf 4 
E om) xS <3 ys > 
2 x a E 4 6 
No response 10.2 16.3 45.9 23.5 3.1 1.0 
Japanese Language, Social Studies 0.2 16.4 51.2 27.5 3.4 1.7 
Mathematics, Science - 15.4 53.8 26.2 3.6 0.8 
Music, Fine Arts 2.2 21.3 40.4 25.0 6.6 4.4 
Physical Education, Health - 17.8 45.0 30.2 4.7 2.3 
Home Economies, Industrial Arts 0.6 21.9 54.8 18.7 1.9 1.9 
Foreigh Language 0.5 13.3 51.7 28.9 4.3 0.9 
Other subjects 122 16.5 47.5 25.0 6.4 3.0 


It can be said that lower Secondary school teachers have a greater degree of awareness of 
this problem than do elementary school teachers. There do not appear to be major subject 
related variations. 


Integrated science course in upper secondary schools 


In the twenty-five years since the adoption 
revisions have radically altered the course 


Seems to have been the increase of enrolm 
upper secondar 


of the new educational system, three successive 


of study. One of the driving forces of the revisions 
ent in schools. As shown in Table 35, enrolment in 


Over 90 per cent by 1974. It is 
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Table 35. Proportion of the age group enrolled in schools. (Expressed as a percentage.) 


[e Fr ee eee 


School level Age range in 1950 in 1974 
SE AE are ee a Se as DEL ra SE Se MS ene Dee 


Kindergarten 3-5 8.9 61.9 


Compulsory 6-14 99.4 99.9 
(Elementary and Lower Secondary) 


Upper Secondary 15-17 42.5 90.8 


Higher education 18-21 10.11 34.7 


eL CD NU Suum m UR E m I oe eie S 


1 
1954. 


In 1973, a new science course called Basic science was created as one of the subjects for 
Upper secondary school science programmes. The course was designed originally for non-science 
Students, to give them a comprehensive idea about the nature of science, the scientific way of 
thinking and the relationship between science and culture. Materials selected from physical, 
chemical, biological and geological sciences are integrated through the concept approach or the 
topic approach. However, the course has been in minimal demand; only 2 per cent of the upper 
Secondary school students took the basic science in 1975, while 29 per cent took physics, 

35 per cent chemistry, 28 per cent biology and 14 per cent earth science. The low demand for 
the basic science course may be related to the question of re-educating teachers for this new course 
and to the need for revising external examination procedures. 


Evaluation of performance in experimentation 


There are many factors contributing to the complexity of practical learning situations. The 
teachers: approach to these factors seems to be oriented more toward synthesis in comparison to 
the Tesearchers who tend to deal with them analytically. 

Recently, great emphasis has been placed on the importance of the "inquiry approach" and 
cess skills" in the teaching of science. It is becoming increasingly popular to have the 
ents actually observe natural objects and phenomena and engage in experimentation and 
Servation as an important part of the learning activity. 

The following is a summary of the aspects of the traditional tests which teachers feel are 
Problematic, They feel that a paper-and-pencil test can be used to evaluate a student's knowledge, 
s is insufficient to evaluate ability and attitude. Furthermore, the student responds to the 
Objective tests only by selecting from the given items. The objective tests are convenient where 
large numbers of answer sheets need to be processed and when results are dealt with statistically. 
But in the case of only one class, students! reports and essays written under little surveillance are 
Probably more effective for obtaining educationally valuable information. 

With these ideas in mind, various kinds of practical research are being carried on. An 
example of a project in the development of performance evaluation undertaken through the close 
co-operation of school teachers and staff members of the Prefectural Education Centres follows. 


"pro 
Stud 
ob 


Finding Out the difference of the concentrations of salt solutions 


First, a Problem such as the one shown below is given to the children to answer in written form 
5 minutes), 


Beakers A, B and C contain salt solutions of varying concentrations. What methods exist 
in ordering the beakers according to decreasing degree of concentration? List as many 
ways as possible to investigate this problem, using a diagram when necessary. 
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A summary of the answer given by fourth, fifth and sixth graders in an elementary school to 
this problem is shown in Table 36. 


Table 36. Proposed methods of differentiating the concentrations of salt solutions; 


difference 
depending on the grades of the children. (Expressed as a percentage.) 
Number The methods proposed by the child Fourth grades Fifth grades Sixth grades 
1 Taste the solutions 94 100 99 
2 Evaporate the solutions and compare 24 15 74 
g the residues 
3 Check the conductivity of electricity s e 60 
of the solutions 
4 Test the buoyancy of objects in the 11 63 39 
solutions 
5 Measure the weights of the solutions 14 42 29 
6 Compare the weights of the solutions 8 28 12 
T Compare the colours of the Solutions 36 63 38 
8 Compare the colour reactions of = " 35 
litmus paper 
9 Pour the solutions over Slugs 26 8 20 
10 Add a fixed amount of salt and compare 11 13 10 
the amounts undissolved 
11 See how much more can be dissolved 3 36 8 
12 Compare the solutions by feeling the 16 1 6 
textures 
13 Use a hydrometer " 1 
14 Other methods 


(16) (18) - 
From Table 36 it is possible 


to see whether the children i 
learning experiences in their School considered this Problem based on 


or in light of their dail e 
Next, the solutions A, B and C are distributed to Feed es peers 
the investigation through actual experimentation. 


experiments are left up to the individual child. 
are to be recorded by the child a. Sses A 
when the children are finished wi recording ps 
Table 37 w. 
Apropos this list, in investigating the relationships b 
Table 37, the following results are obtained. Be between the 
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Proposals and results of experiments. i 


Table 37. 
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proposed in the original investigation. 
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9 Me thods 
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Children 
21 


X Unsuccessful experimentation. 


Ambiguous result through experimentation. 


P Toposals number 8 and 13 are not included in this table. 


1 


193 


Integrated science 4 


The situation is quite similar for girls. That is, the number of methods proposed and the 
scores are related, but there is little correlation between the number of methods proposed and the 


number of successful experiments, or between the number of successful experiments and the 
paper-and-pencil test scores. Thus, it can be said that the learning experience the student 
undergoes through experimentation cannot be valuated by a paper-and-pencil test, and as can be 


seen in this example, the reports made by the students pertaining to the experiment are an important 
basis for evaluation. 


The methods explained above require considerable time and expertise on the part of the teacher 
in analysing the students' reports. 


There are no appropriate ways to quantify objectively the 
results of the evaluation. However, the students! observations provide certain intriguing information. 
Listed below are a few examples. 


When the tasting method was used most children were able to identify the densest solution C, 
but they were divided on the ordering of A and B. 


One child mentioned in his report that 
"The sense of taste of the tongue is unreliable". 


There were some students who only observed residue, while others measured its weight. 
Those who could not discriminate using this method did not use enough solution. 


The methods employing the conductivity of electricity obtained the most accurate results. 
Though 90 per cent of the boys proposed this method, onl 


y 50 per cent of the girls did so. 

The impressions concerning these experiments included such Opinions as: I think it's great 
that we can draw up our own plans for the experiments and use whatever kinds of equipment 
necessary to carry them out (a girl). What I expect will happen does not always come true when 
I actually do it (a boy). 

It is a very rewarding experience since it is possible to find out the students! ideas and points 
of view during the process of forming a list by reading off each student's reports The students 
welcome this method very much, and do not consider themselves being tested. They take avid 
interest in the project matter and actively work on it (a teacher). 

According to the teachers, what they need are examples based on a 
and ways of thinking and methodology Suggested b 
test instruments. 


The individual teachers 
their own system of evaluation. 


ctual experimental results 
rather than mere ready-made 
can take these factors i 


in the integrated science courses. 
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